-- Logs begin at Thu 2022-08-04 09:43:06 UTC, end at Thu 2022-08-04 11:07:53 UTC. -- Aug 04 10:17:33.133264 dv-cinder-852003-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 04 10:17:33.142672 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 04 10:17:33.147189 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Aug 4 10:17:33 2022] *** Aug 04 10:17:33.147189 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 04 10:17:33.147189 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 04 10:17:33.147189 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: nodename: dv-cinder-852003-1-iscsi Aug 04 10:17:33.147189 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: machine: x86_64 Aug 04 10:17:33.147189 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: clock source: unix Aug 04 10:17:33.147189 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: pcre jit disabled Aug 04 10:17:33.147189 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: detected number of CPU cores: 4 Aug 04 10:17:33.147189 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: current working directory: / Aug 04 10:17:33.147189 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: detected binary path: /usr/bin/uwsgi-core Aug 04 10:17:33.147189 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: your processes number limit is 47680 Aug 04 10:17:33.147189 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: your memory page size is 4096 bytes Aug 04 10:17:33.147189 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: detected max file descriptor number: 2048 Aug 04 10:17:33.147189 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: lock engine: pthread robust mutexes Aug 04 10:17:33.147189 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: thunder lock: enabled Aug 04 10:17:33.147189 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 04 10:17:33.147189 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 04 10:17:33.216480 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: Python main interpreter initialized at 0x5559092489c0 Aug 04 10:17:33.216480 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: python threads support enabled Aug 04 10:17:33.216480 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: your server socket listen backlog is limited to 100 connections Aug 04 10:17:33.216480 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: your mercy for graceful operations on workers is 90 seconds Aug 04 10:17:33.217092 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: mapped 403077 bytes (393 KB) for 2 cores Aug 04 10:17:33.217230 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: *** Operational MODE: preforking *** Aug 04 10:17:33.217788 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: *** uWSGI is running in multiple interpreter mode *** Aug 04 10:17:33.217788 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: spawned uWSGI master process (pid: 80227) Aug 04 10:17:33.218152 dv-cinder-852003-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 04 10:17:33.219528 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: spawned uWSGI worker 1 (pid: 80235, cores: 1) Aug 04 10:17:33.222101 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: spawned uWSGI worker 2 (pid: 80236, cores: 1) Aug 04 10:17:33.222101 dv-cinder-852003-1-iscsi devstack@keystone.service[80227]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 04 10:17:38.902837 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 10:17:38.902837 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: from cryptography.utils import int_from_bytes Aug 04 10:17:38.903748 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 10:17:38.903748 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: from cryptography.utils import int_from_bytes Aug 04 10:17:38.958832 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 04 10:17:38.959112 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 04 10:17:38.959211 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] command line args: [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 04 10:17:38.959456 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 04 10:17:38.959647 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ================================================================================ {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 04 10:17:38.959902 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] admin_token = **** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.960185 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] config_dir = [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.960385 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.960553 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] config_source = [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.960838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] debug = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.961073 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.961474 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.961703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.962745 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.962745 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.962745 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] list_limit = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.962745 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] log_config_append = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.962745 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.963279 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] log_dir = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.963279 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] log_file = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.963403 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.963634 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.963917 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.964201 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.964365 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.964533 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.964814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.965114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.965364 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.965649 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.965878 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.966123 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.966381 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.966709 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.967021 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.967561 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.121/identity {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.967820 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] publish_errors = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.968100 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.968355 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.968536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.968819 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.969043 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.969342 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] standard_threads = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.969579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.969850 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.970030 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.970210 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] use_journal = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.970388 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] use_json = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.970565 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] use_stderr = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.970744 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] use_syslog = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.970977 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:38.971321 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.971583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.971857 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.972203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.972459 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.972702 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.972954 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.973193 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.973443 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.973755 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.974040 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.974281 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.974552 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.974791 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.975047 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.975297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.975607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.975873 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.976163 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.976417 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.976692 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.976951 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.977241 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.977525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.977824 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.978063 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.978290 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.978545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.978862 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.979116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.979384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.979616 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.979846 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.980124 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.980379 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.980617 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.980867 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.981101 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.981340 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.981577 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.981811 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.982039 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.982290 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.982499 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.982857 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.983116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.983376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.983613 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.983842 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.984178 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.984415 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.984687 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.984916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.985177 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.985465 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.985722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.985968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.986205 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.986441 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.986772 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.987018 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.987280 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.987511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.987782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.988048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.988288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.988529 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.988764 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.989041 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.989286 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.989890 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.990142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.990383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.990616 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.990859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.991109 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.991345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.991581 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.991827 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.992124 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.992414 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.992651 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.992942 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.993085 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.993366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.993608 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.993850 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.994087 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.994328 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.994564 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.994800 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.995061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.995292 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.995520 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.995723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.995983 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.996219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.996463 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.996728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.996957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.997196 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.997461 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.997724 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.998048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.998195 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] auth.external = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.998531 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.998950 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.999197 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.999438 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] auth.password = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:38.999752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] auth.token = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.000023 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.000325 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.000749 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.000885 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.001233 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.001559 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.001798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.002038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] credential.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.002292 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.002547 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.002799 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.003060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.003379 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.003618 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.003877 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.004165 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.004504 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.004858 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.005159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.005412 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.005774 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.006119 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.006381 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.006618 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] federation.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.006862 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.007213 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.007560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.007923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.008203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.008464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.008802 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.009070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.009351 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.009667 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.009922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.010182 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.010510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.010863 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.010997 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.011233 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.011486 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.011720 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.012036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.012295 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.012553 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.012787 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.013803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.013803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.013803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.013803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.014229 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.014352 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.014567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.014841 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.015081 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.015318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.015550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.015860 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.016135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.016386 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.016693 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.017007 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.017263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.017496 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.017746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.017995 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.018231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.018484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.018728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.018960 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.019226 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.019517 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.019870 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.020141 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.020414 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.020654 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.020892 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.021164 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.021511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.021760 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.021994 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.022314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.022571 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.022828 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.023066 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.023504 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.023720 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.024233 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.119468 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.119777 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.120102 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.120403 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.120676 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.120926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.121167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.121387 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.121632 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.121879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.122122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.122462 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.122721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.122977 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.123231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.123483 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.123724 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.124016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.124262 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.124535 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.124818 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.125081 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.125328 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.125583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.125832 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.126089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.126389 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.126655 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.126909 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.127129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.127393 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.127727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.127999 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.128257 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.128494 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.128740 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] resource.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.129070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.129337 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.129599 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.129857 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.130110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.130426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.130706 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.130932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.131183 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.131487 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] role.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.131743 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] role.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.131997 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.132332 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.132598 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.132890 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.133143 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.133416 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.133745 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.133998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.134272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.134512 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.134764 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.135018 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.135270 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.135521 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.136002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.136248 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.136506 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.136758 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.137017 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.137271 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.137509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.137769 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.137931 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 10:17:39.137931 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: from cryptography.utils import int_from_bytes Aug 04 10:17:39.138746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.138746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.138965 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 10:17:39.138965 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: from cryptography.utils import int_from_bytes Aug 04 10:17:39.139143 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.139143 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.139143 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.139367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.139518 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.139797 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.140120 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.140372 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.140611 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.140847 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] token.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.141086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.141346 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.141575 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.141901 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.142168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.142518 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.142660 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.142905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.143159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.143400 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.143641 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.143878 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.144158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.144415 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.144649 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.144893 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.145200 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.145469 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.connection = **** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.145719 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.145970 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.146211 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.146456 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.146696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.146932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.147175 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.147413 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.147661 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.147899 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.148171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.148523 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.148779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.149018 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.149267 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.149496 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.149739 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.150061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.150240 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 04 10:17:39.182042 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.182297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.182402 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.192598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 04 10:17:39.192848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.193003 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 04 10:17:39.193003 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] command line args: [] {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 04 10:17:39.193293 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 04 10:17:39.193432 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ================================================================================ {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 04 10:17:39.193668 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] admin_token = **** {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.193922 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] config_dir = [] {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.194116 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.194298 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] config_source = [] {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.194484 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] debug = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.194714 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.194951 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.195144 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.195495 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.195909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.196378 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] list_limit = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.196592 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] log_config_append = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.196806 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.197028 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] log_dir = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.197256 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] log_file = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.197598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.197864 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.198123 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.198382 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.198567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.198786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.198993 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.199246 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.199487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.199724 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.199990 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.200236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.200489 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.200745 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.201057 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.201703 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.121/identity {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.201965 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] publish_errors = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.202206 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.202446 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.202636 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.202920 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.203180 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.203519 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] standard_threads = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.203831 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.204179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.204384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.204715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] use_journal = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.204913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] use_json = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.205101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] use_stderr = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.205258 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] use_syslog = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.205532 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 10:17:39.205836 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.206144 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.206539 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.206844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.207251 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.207570 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.207928 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.208253 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.208542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.208796 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.209083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.209357 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.209739 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.210038 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.210324 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.210613 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.211002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.211376 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.211646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.211994 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.212267 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.212631 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.212924 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.213238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.213558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.213891 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.214173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.214435 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.214691 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.215060 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.215322 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.215562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.215807 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.216141 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.216411 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.216655 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.216919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.217202 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.217546 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.217828 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.218126 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.218478 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.218755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.219045 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.219325 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.219650 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.220172 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.220356 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.220540 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.220855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.221123 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.221392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.221629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.221877 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.222127 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.222374 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.222618 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.222862 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.223101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.223244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.223405 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.223559 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.223712 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.223851 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.224064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.224360 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.224590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.224825 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.225099 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.225372 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.225686 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.226023 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.226483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.226825 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.227094 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.227408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.227733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.228018 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.228286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.228568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.228849 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.229096 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.229338 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.229576 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.229828 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.230070 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.230345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.230598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.230850 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.231096 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.231348 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.231593 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.231838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.232111 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.232380 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.232723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.233013 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.233309 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.233579 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.233875 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.234224 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.234538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.234825 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.235092 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.235386 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.235687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.235881 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] auth.external = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.236169 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.236520 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.236808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.237057 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] auth.password = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.237271 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] auth.token = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.237626 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.237934 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.238229 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.238531 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.238799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.239162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.239403 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.240126 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] credential.caching = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.240471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.240735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.240996 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.241253 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.241608 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.241837 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.242135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.242452 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.242805 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.243100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.243449 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.243770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.244153 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.244454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.244839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.245083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] federation.caching = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.245357 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.245676 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.245976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.246270 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.246561 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.246780 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.247053 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.247302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.247569 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.247829 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.248124 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.248370 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] identity.caching = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.248633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.248882 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.249127 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.249370 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.249631 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.249906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.250225 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.250564 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.250881 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.251128 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.251475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.251731 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.252011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.252274 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.252531 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.252795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.253050 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.253333 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.253580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.253825 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.254060 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.254306 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.254546 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.254791 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.255042 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.255299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.255562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.255833 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.256142 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.256476 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.256824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.257083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.257367 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.257636 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.258004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.258347 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.258614 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.258859 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.259133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.259374 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.259648 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.260236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.260525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.260816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.261093 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.261447 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.261744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.262022 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.262308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.262589 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.262935 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.user = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.263244 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.263556 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.263838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.264224 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.264527 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.264830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.265080 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.265355 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.265706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.265995 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.266248 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.266560 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.266922 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.267179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.267466 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.267767 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.268155 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.268422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.268717 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.268991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.269310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.269577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.269903 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.270186 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.270484 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.270734 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.271055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.271392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.271689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.271983 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.272255 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.272683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.272947 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.273202 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.273473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.273725 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.273990 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] resource.caching = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.274257 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.274517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.274763 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.275026 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.275321 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.275636 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.276060 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.276360 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.276616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.276891 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] role.caching = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.277224 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] role.driver = sql {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.277537 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.277850 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.278170 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.278483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.278748 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.279008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.279262 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.279519 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.279778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.280046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.280303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.280555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.280811 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.281069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.281528 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.281797 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.282085 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.282419 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.282784 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.283075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.283345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.283723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.283925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.284097 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.284293 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.284535 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.284779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.285044 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.285304 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.285568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.285860 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.286144 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.286514 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.286788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.287080 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] token.caching = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.287327 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.287584 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.287827 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.288157 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.288512 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.288808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.289152 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.289535 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.289804 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.290079 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.290376 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.290680 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.291032 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.291310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.291548 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.291802 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.292170 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.292560 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] database.connection = **** {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.292862 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.293124 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.293424 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.293774 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.294089 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.294338 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.294588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.294835 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.295088 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.295330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.295654 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.296002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.296244 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.296494 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.296803 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.297089 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.297460 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.297730 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 10:17:39.297941 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80235) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 04 10:17:39.327069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.327387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.327524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.337255 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.338015 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.338248 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.359510 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.359753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.403135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.412042 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.412303 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.412520 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.412738 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.412956 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.413211 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.437529 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.437716 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.437869 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.438111 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.438298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.438541 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.438751 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.443215 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.443517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.450783 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.464722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.464958 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.465161 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.465383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.465630 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.465835 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.466123 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.483416 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.483616 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 10:17:39.484175 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.484409 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 10:17:39.484703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.484962 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.485219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.485472 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.490144 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.490381 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.490610 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.490837 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.491059 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.491280 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.495290 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.495564 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.495834 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.496145 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.503185 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.505072 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.505312 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.505546 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.505781 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.506028 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.506292 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.506424 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.506639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.506796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.507010 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.507340 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.507630 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.508042 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.508514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.508795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.509070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.509350 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.509613 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.515641 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.518874 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.519106 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.519336 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.519563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.519780 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.520023 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.520242 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80235) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 10:17:39.530975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.531190 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 10:17:39.531548 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.531832 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 10:17:39.532090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.532415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.532708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.532815 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.532999 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.534397 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.534680 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.539933 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.540232 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.540507 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.544536 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.544965 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.545362 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.545752 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.547444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.547896 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.552550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.552893 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.553269 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.553550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.553830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.554315 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.554748 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.555069 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.555193 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80235) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.561426 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80235) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.561747 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.562197 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.562584 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.562950 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.563243 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.563622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.564071 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.564619 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.565212 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.565671 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.574912 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.575203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.579622 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.583248 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.583549 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.588531 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.591957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.595052 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.595536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.595851 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.596166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.596452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.596790 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.596910 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80235) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.597084 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.597173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.602751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80235) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.603038 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.603418 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.611426 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80235) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.611714 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.614296 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.614959 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.615243 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.615528 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.616878 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80235) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.617266 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.617661 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.617998 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.618314 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.618736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.619085 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.619383 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.623258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.624865 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.625150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.627840 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.628145 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.628433 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.628715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.628989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.636673 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.636976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.641578 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80235) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.641830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.642191 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.642471 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.642756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.643010 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.643289 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.643546 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.646080 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.646393 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.651698 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80235) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.655544 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80235) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.659090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.659203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.659543 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.660094 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.660391 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.660695 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.660991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.662325 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.665338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.666810 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.668323 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.668579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.668886 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.675777 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.677312 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.679106 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80235) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.679447 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.679829 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.680221 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.680323 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.680601 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.680883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.681161 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.688667 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.689202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.690065 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.690065 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.690377 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.690545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.693359 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80235) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.693617 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.694079 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.694574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.695127 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.696746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.697106 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.697304 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.697578 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.697849 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.698139 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.698416 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.698690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.698969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.699234 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.699499 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.699763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.700059 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.700325 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.708710 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80235) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.709011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.709305 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.709654 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.710051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.710380 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.710799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.730168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80235) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.733706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80235) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.734741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5559092489c0 pid: 80236 (default app) Aug 04 10:17:39.736890 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.738548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de50e5b-d9b2-451d-9e91-d437635f45f5 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:39.738841 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de50e5b-d9b2-451d-9e91-d437635f45f5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:39.739118 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de50e5b-d9b2-451d-9e91-d437635f45f5 None None] PATH_INFO: `/v3` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:39.740021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.741619 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80235) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.742069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.742444 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.745315 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1/1] 10.222.0.121 () {50 vars in 796 bytes} [Thu Aug 4 10:17:39 2022] GET /identity/v3/ => generated 256 bytes in 10 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:17:39.749775 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.751751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80235) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.755310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.755653 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.756066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.756425 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.765043 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80235) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.765342 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.765629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.774318 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80235) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 10:17:39.774663 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.774955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.775234 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.775514 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.775790 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.776101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.776390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.776791 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.777068 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.777345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.777627 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.777899 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.778173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80235) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 10:17:39.815146 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5559092489c0 pid: 80235 (default app) Aug 04 10:17:47.948669 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09138fb-cdb6-4f81-a2d5-7527a626e75e None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:47.949534 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09138fb-cdb6-4f81-a2d5-7527a626e75e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:47.949534 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09138fb-cdb6-4f81-a2d5-7527a626e75e None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:47.950468 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 2/2] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:17:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 10:17:47.958715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7fca6-b3b9-494b-baff-2021d64f7b7d None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:47.959251 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7fca6-b3b9-494b-baff-2021d64f7b7d None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:47.959251 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7fca6-b3b9-494b-baff-2021d64f7b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:48.032261 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: INFO dbcounter [None req-1ea7fca6-b3b9-494b-baff-2021d64f7b7d None None] Registered counter for database keystone Aug 04 10:17:48.053891 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG oslo_db.sqlalchemy.engines [None req-1ea7fca6-b3b9-494b-baff-2021d64f7b7d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80235) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 04 10:17:48.062393 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writer thread running {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 04 10:17:48.261008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG passlib.handlers.bcrypt [None req-1ea7fca6-b3b9-494b-baff-2021d64f7b7d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80235) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 04 10:17:48.261330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG passlib.handlers.bcrypt [None req-1ea7fca6-b3b9-494b-baff-2021d64f7b7d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80235) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 04 10:17:48.321136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-1ea7fca6-b3b9-494b-baff-2021d64f7b7d None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:48.322822 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-1ea7fca6-b3b9-494b-baff-2021d64f7b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:48.476667 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1/3] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:17:47 2022] POST /identity/v3/auth/tokens => generated 915 bytes in 521 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 04 10:17:48.615000 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09138fb-cdb6-4f81-a2d5-7527a626e75e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:48.692037 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: INFO dbcounter [None req-c09138fb-cdb6-4f81-a2d5-7527a626e75e None None] Registered counter for database keystone Aug 04 10:17:48.719311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG oslo_db.sqlalchemy.engines [None req-c09138fb-cdb6-4f81-a2d5-7527a626e75e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80236) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 04 10:17:48.726900 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writer thread running {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 04 10:17:48.859120 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09138fb-cdb6-4f81-a2d5-7527a626e75e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:48.865122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14aa1f-ada1-42a8-bcd7-28a91d863ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:48.865919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14aa1f-ada1-42a8-bcd7-28a91d863ffa admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:48.866256 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14aa1f-ada1-42a8-bcd7-28a91d863ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:48.866540 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14aa1f-ada1-42a8-bcd7-28a91d863ffa admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:48.877191 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb14aa1f-ada1-42a8-bcd7-28a91d863ffa admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:48.932127 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: WARNING keystone.common.rbac_enforcer.enforcer [None req-eb14aa1f-ada1-42a8-bcd7-28a91d863ffa admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 04 10:17:48.932463 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb14aa1f-ada1-42a8-bcd7-28a91d863ffa admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:48.944701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 3/4] 10.222.0.121 () {58 vars in 1329 bytes} [Thu Aug 4 10:17:48 2022] GET /identity/v3/auth/tokens => generated 915 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:17:48.953409 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7fca6-b3b9-494b-baff-2021d64f7b7d None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:48.970184 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7fca6-b3b9-494b-baff-2021d64f7b7d None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:48.977312 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec40a63-47d6-4698-a843-b5e6296b0b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:48.978665 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec40a63-47d6-4698-a843-b5e6296b0b7d admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:48.978903 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec40a63-47d6-4698-a843-b5e6296b0b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:48.979304 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec40a63-47d6-4698-a843-b5e6296b0b7d admin admin] PATH_INFO: `/v3/projects/03f48ab98257418c836fe2e290bdf749` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:48.981100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec40a63-47d6-4698-a843-b5e6296b0b7d admin admin] RBAC: Authorizing `identity:get_project(project_id=03f48ab98257418c836fe2e290bdf749)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:49.037517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0ec40a63-47d6-4698-a843-b5e6296b0b7d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 04 10:17:49.037846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec40a63-47d6-4698-a843-b5e6296b0b7d admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:49.043979 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 2/5] 10.222.0.121 () {56 vars in 1212 bytes} [Thu Aug 4 10:17:48 2022] GET /identity/v3/projects/03f48ab98257418c836fe2e290bdf749 => generated 345 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:17:51.265109 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823a3a3-33e0-4f02-873b-53b24bc747c0 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:51.266084 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823a3a3-33e0-4f02-873b-53b24bc747c0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:51.266084 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823a3a3-33e0-4f02-873b-53b24bc747c0 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:51.269935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 4/6] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:17:51 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:17:51.277099 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351573e6-6e48-4811-9daf-c5bf78b6cc82 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:51.277789 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351573e6-6e48-4811-9daf-c5bf78b6cc82 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:51.277789 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351573e6-6e48-4811-9daf-c5bf78b6cc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:51.305221 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-351573e6-6e48-4811-9daf-c5bf78b6cc82 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:51.307026 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-351573e6-6e48-4811-9daf-c5bf78b6cc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:51.318028 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 3/7] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:17:51 2022] POST /identity/v3/auth/tokens => generated 915 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 04 10:17:51.343924 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823a3a3-33e0-4f02-873b-53b24bc747c0 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:51.359358 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823a3a3-33e0-4f02-873b-53b24bc747c0 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:51.364900 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c633a-a2a9-4e4a-ba73-5e2ba9eaa862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:51.365617 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c633a-a2a9-4e4a-ba73-5e2ba9eaa862 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:51.365921 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c633a-a2a9-4e4a-ba73-5e2ba9eaa862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:51.366201 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c633a-a2a9-4e4a-ba73-5e2ba9eaa862 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:51.383650 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c633a-a2a9-4e4a-ba73-5e2ba9eaa862 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:51.386568 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c633a-a2a9-4e4a-ba73-5e2ba9eaa862 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:51.402392 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 5/8] 10.222.0.121 () {58 vars in 1329 bytes} [Thu Aug 4 10:17:51 2022] GET /identity/v3/auth/tokens => generated 915 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:17:51.415219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351573e6-6e48-4811-9daf-c5bf78b6cc82 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:51.425758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351573e6-6e48-4811-9daf-c5bf78b6cc82 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:51.433111 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd37a2-1b55-4c5e-893f-df52fc1fd8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:51.433111 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd37a2-1b55-4c5e-893f-df52fc1fd8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:51.433465 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd37a2-1b55-4c5e-893f-df52fc1fd8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:51.433558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd37a2-1b55-4c5e-893f-df52fc1fd8d3 admin admin] PATH_INFO: `/v3/users/4d1eaa7a05e745a5817149d8a56dab58` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:51.435232 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dd37a2-1b55-4c5e-893f-df52fc1fd8d3 admin admin] RBAC: Authorizing `identity:get_user(user_id=4d1eaa7a05e745a5817149d8a56dab58)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:51.437201 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dd37a2-1b55-4c5e-893f-df52fc1fd8d3 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:51.438925 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 4/9] 10.222.0.121 () {56 vars in 1203 bytes} [Thu Aug 4 10:17:51 2022] GET /identity/v3/users/4d1eaa7a05e745a5817149d8a56dab58 => generated 248 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:17:53.674968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b80b0-863c-4ae7-90a0-36fa38bbfff0 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:53.675586 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b80b0-863c-4ae7-90a0-36fa38bbfff0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:53.675586 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b80b0-863c-4ae7-90a0-36fa38bbfff0 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:53.676915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 6/10] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:17:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:17:53.682995 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c34135-e719-4e81-9582-848cfadcb75e None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:53.683883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c34135-e719-4e81-9582-848cfadcb75e None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:53.683883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c34135-e719-4e81-9582-848cfadcb75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:53.705008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-74c34135-e719-4e81-9582-848cfadcb75e None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:53.706238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-74c34135-e719-4e81-9582-848cfadcb75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:53.777447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10026264-7a8e-4538-859a-e74b18216d57 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:53.777730 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10026264-7a8e-4538-859a-e74b18216d57 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:53.777972 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10026264-7a8e-4538-859a-e74b18216d57 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:53.779340 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 7/11] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:17:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:17:53.785236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9528e-f151-42b4-826a-0a55a76f0375 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:53.785419 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9528e-f151-42b4-826a-0a55a76f0375 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:53.785648 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9528e-f151-42b4-826a-0a55a76f0375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:53.794847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 5/12] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:17:53 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:17:53.818220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c34135-e719-4e81-9582-848cfadcb75e None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:53.830522 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG passlib.handlers.bcrypt [None req-86e9528e-f151-42b4-826a-0a55a76f0375 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80236) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 04 10:17:53.830956 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG passlib.handlers.bcrypt [None req-86e9528e-f151-42b4-826a-0a55a76f0375 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80236) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 04 10:17:53.843995 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:53.877342 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-86e9528e-f151-42b4-826a-0a55a76f0375 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:53.879199 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-86e9528e-f151-42b4-826a-0a55a76f0375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:53.934844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c34135-e719-4e81-9582-848cfadcb75e None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:53.971774 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932aeea-86b2-42ff-a400-5c604fbcdcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:53.972953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932aeea-86b2-42ff-a400-5c604fbcdcc1 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:53.972953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932aeea-86b2-42ff-a400-5c604fbcdcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:53.973190 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932aeea-86b2-42ff-a400-5c604fbcdcc1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:53.986490 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932aeea-86b2-42ff-a400-5c604fbcdcc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:17:53.987187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 8/13] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:17:53 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:17:53.996454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-5932aeea-86b2-42ff-a400-5c604fbcdcc1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:17:53.996454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:17:53.996454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:17:53.996454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:17:53.996454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:17:53.996454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:17:53.996454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:17:53.996454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:17:53.996454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:17:53.996454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:17:53.996454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:17:53.996454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:17:53.996454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:17:53.996454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:17:53.996454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:17:53.996454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:17:53.997571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:17:53.997571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:17:53.997571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:17:53.997571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:17:53.997571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:17:53.997571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:17:53.997571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:17:53.997571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:17:53.997571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:17:53.997571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:17:53.997571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:17:53.997571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:17:53.997571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:17:53.997571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:17:53.997571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:17:53.997571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:17:53.998707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:17:53.998707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:17:53.998707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:17:53.998707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:17:53.998707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:17:53.998707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:17:53.998707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:17:53.998707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:17:53.998707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:17:53.998707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:17:53.998707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:17:53.998707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:17:53.998707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 6/14] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:17:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:17:54.003934 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9528e-f151-42b4-826a-0a55a76f0375 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:54.010547 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932aeea-86b2-42ff-a400-5c604fbcdcc1 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:54.102236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:54.118165 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9528e-f151-42b4-826a-0a55a76f0375 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:54.120889 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:54.124176 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932aeea-86b2-42ff-a400-5c604fbcdcc1 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:54.159151 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e189c-1c0f-45d2-8307-14518945cfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:54.160121 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e189c-1c0f-45d2-8307-14518945cfbc None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:54.160413 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e189c-1c0f-45d2-8307-14518945cfbc None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:54.160687 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e189c-1c0f-45d2-8307-14518945cfbc None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:54.161646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e189c-1c0f-45d2-8307-14518945cfbc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:17:54.161949 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e189c-1c0f-45d2-8307-14518945cfbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:54.163586 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e189c-1c0f-45d2-8307-14518945cfbc None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:54.171537 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a7f37-98d4-45af-a155-ae4911da4673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:54.172210 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a7f37-98d4-45af-a155-ae4911da4673 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:54.172501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a7f37-98d4-45af-a155-ae4911da4673 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:54.172784 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a7f37-98d4-45af-a155-ae4911da4673 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:54.178492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 9/15] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:17:54 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:17:54.185646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e189c-1c0f-45d2-8307-14518945cfbc None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:54.253167 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a7f37-98d4-45af-a155-ae4911da4673 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:54.254747 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a7f37-98d4-45af-a155-ae4911da4673 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:54.301372 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e189c-1c0f-45d2-8307-14518945cfbc None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:54.337754 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29659548-e522-400f-a887-24bac9c359ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:54.338696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29659548-e522-400f-a887-24bac9c359ea None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:54.338985 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29659548-e522-400f-a887-24bac9c359ea None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:54.339264 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29659548-e522-400f-a887-24bac9c359ea None admin] PATH_INFO: `/v3/users/4d1eaa7a05e745a5817149d8a56dab58` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:54.341060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29659548-e522-400f-a887-24bac9c359ea None admin] RBAC: Authorizing `identity:get_user(user_id=4d1eaa7a05e745a5817149d8a56dab58)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:54.342606 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29659548-e522-400f-a887-24bac9c359ea None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:54.344480 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 10/16] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:17:54 2022] GET /identity/v3/users/4d1eaa7a05e745a5817149d8a56dab58 => generated 248 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:17:54.350626 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29659548-e522-400f-a887-24bac9c359ea None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:54.400639 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:54.461133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:54.478520 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29659548-e522-400f-a887-24bac9c359ea None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:54.487967 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 7/17] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:17:54 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:17:54.494756 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a7f37-98d4-45af-a155-ae4911da4673 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:54.523605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7141fe1-366b-4445-8b11-58624abaca34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:54.524312 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7141fe1-366b-4445-8b11-58624abaca34 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:54.524603 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7141fe1-366b-4445-8b11-58624abaca34 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:54.524913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7141fe1-366b-4445-8b11-58624abaca34 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:54.526157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7141fe1-366b-4445-8b11-58624abaca34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:54.527671 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7141fe1-366b-4445-8b11-58624abaca34 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:54.529367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 11/18] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:17:54 2022] GET /identity/v3/domains/default => generated 200 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:17:54.535667 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7141fe1-366b-4445-8b11-58624abaca34 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:54.625972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a7f37-98d4-45af-a155-ae4911da4673 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:54.652550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7141fe1-366b-4445-8b11-58624abaca34 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:54.654081 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:54.661985 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5dcaa9-d5c2-4200-9e36-290abdf28d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:54.662666 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5dcaa9-d5c2-4200-9e36-290abdf28d2d None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:54.662958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5dcaa9-d5c2-4200-9e36-290abdf28d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:54.663240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5dcaa9-d5c2-4200-9e36-290abdf28d2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:54.674809 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5dcaa9-d5c2-4200-9e36-290abdf28d2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:54.676558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5dcaa9-d5c2-4200-9e36-290abdf28d2d None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:54.688219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3746dd-eb6c-469d-88b8-7b6b057c1aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:54.688781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3746dd-eb6c-469d-88b8-7b6b057c1aaa None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:54.689071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3746dd-eb6c-469d-88b8-7b6b057c1aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:54.689354 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3746dd-eb6c-469d-88b8-7b6b057c1aaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:54.690203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3746dd-eb6c-469d-88b8-7b6b057c1aaa None admin] RBAC: Adding query filter params (role.id=7342106a255b4bce8f150726602a8b95, scope.domain.id=default, user.id=4d1eaa7a05e745a5817149d8a56dab58) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:17:54.690520 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3746dd-eb6c-469d-88b8-7b6b057c1aaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:54.692044 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3746dd-eb6c-469d-88b8-7b6b057c1aaa None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:54.694070 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-6c5dcaa9-d5c2-4200-9e36-290abdf28d2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:17:54.694070 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:17:54.694070 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:17:54.694070 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:17:54.694070 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:17:54.694070 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:17:54.694070 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:17:54.694070 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:17:54.694070 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:17:54.694070 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:17:54.694070 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:17:54.694070 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:17:54.694070 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:17:54.694070 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:17:54.694070 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:17:54.695228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:17:54.695228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:17:54.695228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:17:54.695228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:17:54.695228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:17:54.695228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:17:54.695228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:17:54.695228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:17:54.695228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:17:54.695228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:17:54.695228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:17:54.695228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:17:54.695228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:17:54.695228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:17:54.695228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:17:54.695228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:17:54.695228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:17:54.696083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:17:54.696083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:17:54.696083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:17:54.696083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:17:54.696083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 8/19] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:17:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:17:54.701488 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5dcaa9-d5c2-4200-9e36-290abdf28d2d None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:54.702459 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 12/20] 10.222.0.121 () {56 vars in 1327 bytes} [Thu Aug 4 10:17:54 2022] GET /identity/v3/role_assignments?user.id=4d1eaa7a05e745a5817149d8a56dab58&scope.domain.id=default&role.id=7342106a255b4bce8f150726602a8b95 => generated 234 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:17:54.746585 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:54.827588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5dcaa9-d5c2-4200-9e36-290abdf28d2d None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:54.866731 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f88a1-63e8-43c8-830a-a1fb66b0a212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:54.867579 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f88a1-63e8-43c8-830a-a1fb66b0a212 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:54.867814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f88a1-63e8-43c8-830a-a1fb66b0a212 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:54.868109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f88a1-63e8-43c8-830a-a1fb66b0a212 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:54.868880 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6f88a1-63e8-43c8-830a-a1fb66b0a212 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:17:54.869188 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6f88a1-63e8-43c8-830a-a1fb66b0a212 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:54.870626 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6f88a1-63e8-43c8-830a-a1fb66b0a212 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:54.889976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 9/21] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:17:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:17:56.867276 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ff6e4-46b6-4379-8781-9f29100c47b7 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:56.867849 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ff6e4-46b6-4379-8781-9f29100c47b7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:56.867849 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ff6e4-46b6-4379-8781-9f29100c47b7 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:56.869131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 13/22] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:17:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:17:56.875556 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f137cf-d425-40ea-b7f4-a62557713ba2 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:56.876138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f137cf-d425-40ea-b7f4-a62557713ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:56.876138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f137cf-d425-40ea-b7f4-a62557713ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:56.901459 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-e2f137cf-d425-40ea-b7f4-a62557713ba2 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:56.902709 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-e2f137cf-d425-40ea-b7f4-a62557713ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:57.004954 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 10/23] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:17:56 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:17:57.047476 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ff6e4-46b6-4379-8781-9f29100c47b7 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:57.226744 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ff6e4-46b6-4379-8781-9f29100c47b7 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:57.256542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5039d-848a-4001-9956-fdfdd196925a None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:57.256926 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5039d-848a-4001-9956-fdfdd196925a None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:57.257129 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5039d-848a-4001-9956-fdfdd196925a None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:57.258895 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 11/24] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:17:57 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:17:57.279168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b730e-469e-4f84-9fce-d2f1dced6971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:57.280277 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b730e-469e-4f84-9fce-d2f1dced6971 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:57.280767 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b730e-469e-4f84-9fce-d2f1dced6971 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:57.281262 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b730e-469e-4f84-9fce-d2f1dced6971 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:57.293549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772057a4-df6c-4c10-acb5-5383b74b8d10 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:57.294067 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772057a4-df6c-4c10-acb5-5383b74b8d10 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:57.294170 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772057a4-df6c-4c10-acb5-5383b74b8d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:57.304300 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455b730e-469e-4f84-9fce-d2f1dced6971 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:17:57.316001 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-455b730e-469e-4f84-9fce-d2f1dced6971 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:17:57.316001 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:17:57.316001 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:17:57.316001 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:17:57.316001 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:17:57.316001 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:17:57.316001 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:17:57.316001 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:17:57.316001 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:17:57.316001 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:17:57.316001 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:17:57.316001 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:17:57.316001 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:17:57.316001 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:17:57.316001 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:17:57.316001 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:17:57.316957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:17:57.316957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:17:57.316957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:17:57.316957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:17:57.316957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:17:57.316957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:17:57.316957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:17:57.316957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:17:57.316957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:17:57.316957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:17:57.316957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:17:57.316957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:17:57.316957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:17:57.316957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:17:57.316957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:17:57.316957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:17:57.317740 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:17:57.317740 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:17:57.317740 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:17:57.317740 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:17:57.317740 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:17:57.317740 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:17:57.317740 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:17:57.317740 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:17:57.317740 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:17:57.317740 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:17:57.317740 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:17:57.317740 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:17:57.318337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-772057a4-df6c-4c10-acb5-5383b74b8d10 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:57.318950 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-772057a4-df6c-4c10-acb5-5383b74b8d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:57.321463 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 14/25] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:17:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:17:57.340177 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b730e-469e-4f84-9fce-d2f1dced6971 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:57.361942 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:57.399782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:57.473781 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 12/26] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:17:57 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:17:57.497747 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772057a4-df6c-4c10-acb5-5383b74b8d10 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:57.533492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b730e-469e-4f84-9fce-d2f1dced6971 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:57.608884 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff734d-c80b-485c-8cfc-0b7e7424efa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:57.609858 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff734d-c80b-485c-8cfc-0b7e7424efa5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:57.610504 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff734d-c80b-485c-8cfc-0b7e7424efa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:57.610934 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff734d-c80b-485c-8cfc-0b7e7424efa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:57.612908 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff734d-c80b-485c-8cfc-0b7e7424efa5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:17:57.613566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff734d-c80b-485c-8cfc-0b7e7424efa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:57.616675 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff734d-c80b-485c-8cfc-0b7e7424efa5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:57.630876 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 15/27] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:17:57 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:17:57.634003 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9396bfb-83c7-4a83-a9a3-bd5edd56ccc3 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:57.637563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9396bfb-83c7-4a83-a9a3-bd5edd56ccc3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:57.639167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9396bfb-83c7-4a83-a9a3-bd5edd56ccc3 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:57.642361 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 16/28] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:17:57 2022] GET /identity => generated 271 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:17:57.644695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9396bfb-83c7-4a83-a9a3-bd5edd56ccc3 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:57.705294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772057a4-df6c-4c10-acb5-5383b74b8d10 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:57.760817 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14765fee-b8c4-41d9-84a1-49fa13d209fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:57.761415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14765fee-b8c4-41d9-84a1-49fa13d209fe None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:57.761701 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14765fee-b8c4-41d9-84a1-49fa13d209fe None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:57.761978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14765fee-b8c4-41d9-84a1-49fa13d209fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:57.774040 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:57.788879 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14765fee-b8c4-41d9-84a1-49fa13d209fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:57.791265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14765fee-b8c4-41d9-84a1-49fa13d209fe None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:57.800330 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9396bfb-83c7-4a83-a9a3-bd5edd56ccc3 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:57.816595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-14765fee-b8c4-41d9-84a1-49fa13d209fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:17:57.816595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:17:57.816595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:17:57.816595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:17:57.816595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:17:57.816595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:17:57.816595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:17:57.816595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:17:57.816595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:17:57.816595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:17:57.816595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:17:57.816595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:17:57.816595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:17:57.816595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:17:57.816595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:17:57.817541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:17:57.817541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:17:57.817541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:17:57.817541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:17:57.817541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:17:57.817541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:17:57.817541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:17:57.817541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:17:57.817541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:17:57.817541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:17:57.817541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:17:57.817541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:17:57.817541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:17:57.817541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:17:57.817541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:17:57.817541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:17:57.817541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:17:57.818405 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:17:57.818405 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:17:57.818405 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:17:57.818405 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:17:57.830623 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 13/29] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:17:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:17:57.832600 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de9822-bea8-4f68-a671-60b2790a7a9b None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:57.835911 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de9822-bea8-4f68-a671-60b2790a7a9b None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:57.836515 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de9822-bea8-4f68-a671-60b2790a7a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:57.863811 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:57.875522 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-f7de9822-bea8-4f68-a671-60b2790a7a9b None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:57.877483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-f7de9822-bea8-4f68-a671-60b2790a7a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:57.901461 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734e96b-8bac-48a3-9b7c-0e5458d4194f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:57.903194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734e96b-8bac-48a3-9b7c-0e5458d4194f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:57.903718 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734e96b-8bac-48a3-9b7c-0e5458d4194f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:57.904317 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734e96b-8bac-48a3-9b7c-0e5458d4194f None admin] PATH_INFO: `/v3/users/4d1eaa7a05e745a5817149d8a56dab58` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:57.907515 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734e96b-8bac-48a3-9b7c-0e5458d4194f None admin] RBAC: Authorizing `identity:get_user(user_id=4d1eaa7a05e745a5817149d8a56dab58)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:57.915409 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734e96b-8bac-48a3-9b7c-0e5458d4194f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:57.919692 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 17/30] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:17:57 2022] GET /identity/v3/users/4d1eaa7a05e745a5817149d8a56dab58 => generated 248 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:17:57.920868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734e96b-8bac-48a3-9b7c-0e5458d4194f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:57.986990 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 14/31] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:17:57 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:17:57.987630 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de9822-bea8-4f68-a671-60b2790a7a9b None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:58.129762 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de9822-bea8-4f68-a671-60b2790a7a9b None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:58.157872 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734e96b-8bac-48a3-9b7c-0e5458d4194f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:58.173478 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d392d-a12e-470a-b640-8df45dbf8f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:58.174331 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d392d-a12e-470a-b640-8df45dbf8f20 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:58.174843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d392d-a12e-470a-b640-8df45dbf8f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:58.175368 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d392d-a12e-470a-b640-8df45dbf8f20 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:58.177845 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d392d-a12e-470a-b640-8df45dbf8f20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:58.179580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d392d-a12e-470a-b640-8df45dbf8f20 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:58.181374 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 15/32] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:17:57 2022] GET /identity/v3/domains/default => generated 200 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:17:58.182164 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d392d-a12e-470a-b640-8df45dbf8f20 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:58.188328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:58.257556 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffe64f-ae97-40ef-8442-31a679a4bac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:58.258221 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffe64f-ae97-40ef-8442-31a679a4bac9 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:58.258510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffe64f-ae97-40ef-8442-31a679a4bac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:58.258790 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffe64f-ae97-40ef-8442-31a679a4bac9 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:58.259521 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ffe64f-ae97-40ef-8442-31a679a4bac9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:17:58.259836 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ffe64f-ae97-40ef-8442-31a679a4bac9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:58.261696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ffe64f-ae97-40ef-8442-31a679a4bac9 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:58.293498 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 18/33] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:17:57 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:17:58.294429 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffe64f-ae97-40ef-8442-31a679a4bac9 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:58.308762 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d392d-a12e-470a-b640-8df45dbf8f20 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:58.324409 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:58.346455 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6dfc4-f302-4991-96a2-368829393f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:58.347107 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6dfc4-f302-4991-96a2-368829393f39 None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:58.347388 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6dfc4-f302-4991-96a2-368829393f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:58.347661 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6dfc4-f302-4991-96a2-368829393f39 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:58.348434 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca6dfc4-f302-4991-96a2-368829393f39 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:58.349872 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca6dfc4-f302-4991-96a2-368829393f39 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:58.425128 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffe64f-ae97-40ef-8442-31a679a4bac9 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:58.464690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad4da99-5397-41bf-9b9c-9066be5c027f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:58.465406 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4da99-5397-41bf-9b9c-9066be5c027f None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:58.465689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4da99-5397-41bf-9b9c-9066be5c027f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:58.465974 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4da99-5397-41bf-9b9c-9066be5c027f None admin] PATH_INFO: `/v3/domains/default/users/4d1eaa7a05e745a5817149d8a56dab58/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:58.473232 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad4da99-5397-41bf-9b9c-9066be5c027f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=4d1eaa7a05e745a5817149d8a56dab58, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:58.474892 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad4da99-5397-41bf-9b9c-9066be5c027f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:58.507878 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 16/34] 10.222.0.121 () {60 vars in 1136 bytes} [Thu Aug 4 10:17:58 2022] POST /identity/v3/roles => generated 220 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:17:58.509361 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaaffd3-2d88-456f-89dc-c6d3c9c8fee1 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:58.509745 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaaffd3-2d88-456f-89dc-c6d3c9c8fee1 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:58.510109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaaffd3-2d88-456f-89dc-c6d3c9c8fee1 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:58.511636 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 17/35] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:17:58 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:17:58.512578 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaaffd3-2d88-456f-89dc-c6d3c9c8fee1 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:58.680356 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:58.753950 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaaffd3-2d88-456f-89dc-c6d3c9c8fee1 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:58.799299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61e395-94fd-4be2-a111-6e7b8feddeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:58.800476 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61e395-94fd-4be2-a111-6e7b8feddeb5 None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:58.801024 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61e395-94fd-4be2-a111-6e7b8feddeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:58.801535 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61e395-94fd-4be2-a111-6e7b8feddeb5 None admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:58.802489 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61e395-94fd-4be2-a111-6e7b8feddeb5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:58.804855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61e395-94fd-4be2-a111-6e7b8feddeb5 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:58.844477 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 18/36] 10.222.0.121 () {60 vars in 1146 bytes} [Thu Aug 4 10:17:58 2022] POST /identity/v3/projects => generated 302 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:17:58.846315 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f223b64-01e2-4f9b-a6ad-1002f535d44e None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:58.846822 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f223b64-01e2-4f9b-a6ad-1002f535d44e None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:58.847333 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f223b64-01e2-4f9b-a6ad-1002f535d44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:58.868554 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 19/37] 10.222.0.121 () {58 vars in 1366 bytes} [Thu Aug 4 10:17:58 2022] PUT /identity/v3/domains/default/users/4d1eaa7a05e745a5817149d8a56dab58/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:17:58.888078 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-9f223b64-01e2-4f9b-a6ad-1002f535d44e None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:58.889804 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-9f223b64-01e2-4f9b-a6ad-1002f535d44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:59.024592 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 19/38] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:17:58 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:17:59.026565 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8d2b7-c928-48e9-9f24-7d6d9c2458e1 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:59.027564 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8d2b7-c928-48e9-9f24-7d6d9c2458e1 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:59.028118 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8d2b7-c928-48e9-9f24-7d6d9c2458e1 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:59.035307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 20/39] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:17:59 2022] GET /identity => generated 271 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:17:59.042316 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2495d-7833-4c6a-bf55-77af92c56c02 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:59.043019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2495d-7833-4c6a-bf55-77af92c56c02 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:59.043554 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2495d-7833-4c6a-bf55-77af92c56c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:59.060151 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8d2b7-c928-48e9-9f24-7d6d9c2458e1 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:59.078273 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-aed2495d-7833-4c6a-bf55-77af92c56c02 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:59.080074 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-aed2495d-7833-4c6a-bf55-77af92c56c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:59.149018 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:59.181849 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 20/40] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:17:59 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:17:59.208104 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed2495d-7833-4c6a-bf55-77af92c56c02 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:59.209891 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:59.238262 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8d2b7-c928-48e9-9f24-7d6d9c2458e1 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:59.281872 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41cf7ae-7776-4b55-a23a-e0ef085fa0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:59.282430 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41cf7ae-7776-4b55-a23a-e0ef085fa0ae None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:59.282716 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41cf7ae-7776-4b55-a23a-e0ef085fa0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:59.283001 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41cf7ae-7776-4b55-a23a-e0ef085fa0ae None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:59.283766 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41cf7ae-7776-4b55-a23a-e0ef085fa0ae None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:59.285388 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41cf7ae-7776-4b55-a23a-e0ef085fa0ae None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:59.304453 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 21/41] 10.222.0.121 () {60 vars in 1136 bytes} [Thu Aug 4 10:17:59 2022] POST /identity/v3/roles => generated 226 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:17:59.305488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da72f82-1306-42b5-9672-ae5d165a8d70 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:59.305787 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da72f82-1306-42b5-9672-ae5d165a8d70 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:59.306070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da72f82-1306-42b5-9672-ae5d165a8d70 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:59.307554 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 22/42] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:17:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:17:59.313438 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad6580-b585-4840-85aa-1ee2211dbee1 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:59.313741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad6580-b585-4840-85aa-1ee2211dbee1 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:59.314097 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad6580-b585-4840-85aa-1ee2211dbee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:59.335283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c6ad6580-b585-4840-85aa-1ee2211dbee1 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:17:59.337089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c6ad6580-b585-4840-85aa-1ee2211dbee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:17:59.368613 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed2495d-7833-4c6a-bf55-77af92c56c02 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:59.419502 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04721534-411f-4cff-ad7c-ece5d29679d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:59.420742 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04721534-411f-4cff-ad7c-ece5d29679d5 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:59.421322 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04721534-411f-4cff-ad7c-ece5d29679d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:59.421611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04721534-411f-4cff-ad7c-ece5d29679d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:59.423040 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04721534-411f-4cff-ad7c-ece5d29679d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:59.424754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04721534-411f-4cff-ad7c-ece5d29679d5 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:59.425838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 23/43] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:17:59 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:17:59.431532 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 21/44] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:17:59 2022] GET /identity/v3/domains/default => generated 200 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:17:59.432818 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad6580-b585-4840-85aa-1ee2211dbee1 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:59.451412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04721534-411f-4cff-ad7c-ece5d29679d5 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:17:59.517733 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:59.520542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:17:59.568628 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad6580-b585-4840-85aa-1ee2211dbee1 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:59.607700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04721534-411f-4cff-ad7c-ece5d29679d5 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:17:59.641574 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae25f2d-077d-47ab-82f5-a6551cd9c6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:59.642363 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae25f2d-077d-47ab-82f5-a6551cd9c6ae None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:59.642531 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae25f2d-077d-47ab-82f5-a6551cd9c6ae None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:59.642994 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae25f2d-077d-47ab-82f5-a6551cd9c6ae None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:59.643821 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae25f2d-077d-47ab-82f5-a6551cd9c6ae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:59.645723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae25f2d-077d-47ab-82f5-a6551cd9c6ae None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:59.650311 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a0ef27-0562-483c-8fd2-6573352445eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:17:59.650975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0ef27-0562-483c-8fd2-6573352445eb None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:17:59.651264 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0ef27-0562-483c-8fd2-6573352445eb None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:17:59.651546 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0ef27-0562-483c-8fd2-6573352445eb None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:17:59.652339 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a0ef27-0562-483c-8fd2-6573352445eb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:17:59.653768 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a0ef27-0562-483c-8fd2-6573352445eb None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:17:59.672373 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 22/45] 10.222.0.121 () {60 vars in 1136 bytes} [Thu Aug 4 10:17:59 2022] POST /identity/v3/roles => generated 224 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:17:59.700042 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 24/46] 10.222.0.121 () {60 vars in 1146 bytes} [Thu Aug 4 10:17:59 2022] POST /identity/v3/projects => generated 313 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:18:01.133373 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cd0a4-f4aa-40ca-983e-57d988a2e1d7 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:01.133992 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cd0a4-f4aa-40ca-983e-57d988a2e1d7 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:01.133992 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cd0a4-f4aa-40ca-983e-57d988a2e1d7 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:01.135343 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 23/47] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:01.141020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc492e-0c4f-41e9-abf1-d869ed6daa48 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:01.141493 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc492e-0c4f-41e9-abf1-d869ed6daa48 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:01.141672 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc492e-0c4f-41e9-abf1-d869ed6daa48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:01.162212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b0cc492e-0c4f-41e9-abf1-d869ed6daa48 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:01.163468 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b0cc492e-0c4f-41e9-abf1-d869ed6daa48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:01.234564 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 25/48] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:01 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:18:01.258586 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cd0a4-f4aa-40ca-983e-57d988a2e1d7 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:01.370826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cd0a4-f4aa-40ca-983e-57d988a2e1d7 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:01.373257 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:01.408893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653af019-5d26-4303-b570-e3f199dd7792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:01.409296 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653af019-5d26-4303-b570-e3f199dd7792 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:01.409582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653af019-5d26-4303-b570-e3f199dd7792 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:01.409898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653af019-5d26-4303-b570-e3f199dd7792 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:01.420711 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653af019-5d26-4303-b570-e3f199dd7792 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:01.425844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-653af019-5d26-4303-b570-e3f199dd7792 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:18:01.425844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:01.425844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:01.425844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:01.425844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:01.425844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:01.425844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:01.425844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:01.425844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:01.425844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:01.425844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:01.425844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:01.425844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:01.425844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:01.425844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:01.425844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:01.426741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:01.426741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:01.426741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:01.426741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:01.426741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:01.426741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:01.426741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:01.426741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:01.426741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:01.426741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:01.426741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:01.426741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:01.426741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:01.426741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:01.426741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:01.426741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:01.427721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:01.427721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:01.427721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:01.427721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:01.427721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:01.427721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:01.427721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:01.427721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:01.427721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:01.427721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:01.427721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:18:01.427721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:01.427721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 24/49] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:18:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:18:01.433578 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc492e-0c4f-41e9-abf1-d869ed6daa48 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:01.493831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:01.563610 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc492e-0c4f-41e9-abf1-d869ed6daa48 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:01.608023 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dad573-0817-49b7-a0eb-e3326dc5911c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:01.608675 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dad573-0817-49b7-a0eb-e3326dc5911c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:01.608966 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dad573-0817-49b7-a0eb-e3326dc5911c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:01.609246 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dad573-0817-49b7-a0eb-e3326dc5911c None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:01.609990 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dad573-0817-49b7-a0eb-e3326dc5911c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:01.610298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dad573-0817-49b7-a0eb-e3326dc5911c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:01.611725 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dad573-0817-49b7-a0eb-e3326dc5911c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:01.620701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 26/50] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:01 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:01.626460 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653af019-5d26-4303-b570-e3f199dd7792 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:01.731740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653af019-5d26-4303-b570-e3f199dd7792 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:01.767767 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a99b65-afa6-429c-ade9-2ec0b98248df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:01.768463 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a99b65-afa6-429c-ade9-2ec0b98248df None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:01.768748 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a99b65-afa6-429c-ade9-2ec0b98248df None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:01.769028 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a99b65-afa6-429c-ade9-2ec0b98248df None admin] PATH_INFO: `/v3/users/4d1eaa7a05e745a5817149d8a56dab58` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:01.770287 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a99b65-afa6-429c-ade9-2ec0b98248df None admin] RBAC: Authorizing `identity:get_user(user_id=4d1eaa7a05e745a5817149d8a56dab58)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:01.771787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a99b65-afa6-429c-ade9-2ec0b98248df None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:01.773519 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 25/51] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:01 2022] GET /identity/v3/users/4d1eaa7a05e745a5817149d8a56dab58 => generated 248 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:01.779339 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dad573-0817-49b7-a0eb-e3326dc5911c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:01.791040 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cc55e-046c-4b3a-9267-7265823e9d9a None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:01.791330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cc55e-046c-4b3a-9267-7265823e9d9a None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:01.791613 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cc55e-046c-4b3a-9267-7265823e9d9a None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:01.793086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 26/52] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 10:18:01.798205 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41d194-ee69-40b3-957c-7c14da3e32a3 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:01.798493 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41d194-ee69-40b3-957c-7c14da3e32a3 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:01.798771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41d194-ee69-40b3-957c-7c14da3e32a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:01.817756 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-da41d194-ee69-40b3-957c-7c14da3e32a3 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:01.819048 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-da41d194-ee69-40b3-957c-7c14da3e32a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:01.847988 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:01.888455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dad573-0817-49b7-a0eb-e3326dc5911c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:01.891882 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 27/53] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:01 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:18:01.899981 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:01.916278 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41d194-ee69-40b3-957c-7c14da3e32a3 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:01.928682 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d21f7-5bd9-43eb-b94c-b27bea0b7940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:01.929399 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d21f7-5bd9-43eb-b94c-b27bea0b7940 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:01.929722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d21f7-5bd9-43eb-b94c-b27bea0b7940 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:01.930009 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d21f7-5bd9-43eb-b94c-b27bea0b7940 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:01.931226 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d21f7-5bd9-43eb-b94c-b27bea0b7940 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:01.932770 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d21f7-5bd9-43eb-b94c-b27bea0b7940 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:01.934407 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 27/54] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:18:01 2022] GET /identity/v3/domains/default => generated 200 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:01.940339 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d21f7-5bd9-43eb-b94c-b27bea0b7940 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:02.021436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41d194-ee69-40b3-957c-7c14da3e32a3 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:02.039514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d21f7-5bd9-43eb-b94c-b27bea0b7940 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:02.059418 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82a8ec-f6cc-4f59-9cec-df8db8b877e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:02.060146 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82a8ec-f6cc-4f59-9cec-df8db8b877e5 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:02.060430 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82a8ec-f6cc-4f59-9cec-df8db8b877e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:02.060707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82a8ec-f6cc-4f59-9cec-df8db8b877e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:02.061892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82a8ec-f6cc-4f59-9cec-df8db8b877e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:02.063399 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82a8ec-f6cc-4f59-9cec-df8db8b877e5 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:02.065143 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 28/55] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:18:01 2022] GET /identity/v3/domains/default => generated 200 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:02.071304 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82a8ec-f6cc-4f59-9cec-df8db8b877e5 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:02.081409 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231816c2-7059-42af-8810-226965cf84a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:02.082063 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231816c2-7059-42af-8810-226965cf84a1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:02.082356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231816c2-7059-42af-8810-226965cf84a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:02.082639 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231816c2-7059-42af-8810-226965cf84a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:02.083488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231816c2-7059-42af-8810-226965cf84a1 None admin] RBAC: Adding query filter params (role.id=7342106a255b4bce8f150726602a8b95, scope.domain.id=default, user.id=4d1eaa7a05e745a5817149d8a56dab58) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:02.083806 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231816c2-7059-42af-8810-226965cf84a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:02.085323 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231816c2-7059-42af-8810-226965cf84a1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:02.091791 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 28/56] 10.222.0.121 () {56 vars in 1327 bytes} [Thu Aug 4 10:18:01 2022] GET /identity/v3/role_assignments?user.id=4d1eaa7a05e745a5817149d8a56dab58&scope.domain.id=default&role.id=7342106a255b4bce8f150726602a8b95 => generated 532 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:18:02.101959 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:02.186681 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82a8ec-f6cc-4f59-9cec-df8db8b877e5 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:02.226269 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f8909-1530-4dc6-809f-de24057b53e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:02.226979 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f8909-1530-4dc6-809f-de24057b53e9 None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:02.227249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f8909-1530-4dc6-809f-de24057b53e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:02.227568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f8909-1530-4dc6-809f-de24057b53e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:02.228508 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f8909-1530-4dc6-809f-de24057b53e9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:02.230037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f8909-1530-4dc6-809f-de24057b53e9 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:02.260944 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 29/57] 10.222.0.121 () {60 vars in 1145 bytes} [Thu Aug 4 10:18:02 2022] POST /identity/v3/projects => generated 299 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:18:04.375471 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1ae67-4c92-4c4a-94d4-a23573a10986 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:04.376194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1ae67-4c92-4c4a-94d4-a23573a10986 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:04.376194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1ae67-4c92-4c4a-94d4-a23573a10986 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:04.377449 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 29/58] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:04.383148 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e7e63-ed3c-42c0-a45b-8b72e50caa4d None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:04.383612 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e7e63-ed3c-42c0-a45b-8b72e50caa4d None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:04.383723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e7e63-ed3c-42c0-a45b-8b72e50caa4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:04.404834 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-727e7e63-ed3c-42c0-a45b-8b72e50caa4d None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:04.406301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-727e7e63-ed3c-42c0-a45b-8b72e50caa4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:04.490291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 30/59] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:04 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:04.518034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1ae67-4c92-4c4a-94d4-a23573a10986 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:04.570181 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:04.629849 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1ae67-4c92-4c4a-94d4-a23573a10986 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:04.664342 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c7831-457e-4bc4-b10e-41711d28a949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:04.665017 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c7831-457e-4bc4-b10e-41711d28a949 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:04.665334 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c7831-457e-4bc4-b10e-41711d28a949 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:04.665613 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c7831-457e-4bc4-b10e-41711d28a949 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:04.666891 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c7831-457e-4bc4-b10e-41711d28a949 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:04.668434 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c7831-457e-4bc4-b10e-41711d28a949 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:04.670052 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 30/60] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:18:04 2022] GET /identity/v3/domains/default => generated 200 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:04.676155 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e7e63-ed3c-42c0-a45b-8b72e50caa4d None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:04.690842 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:04.780980 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e7e63-ed3c-42c0-a45b-8b72e50caa4d None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:04.816592 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7e725-ee8e-4e1c-b420-24b7b5dd51e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:04.817221 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7e725-ee8e-4e1c-b420-24b7b5dd51e6 None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:04.817524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7e725-ee8e-4e1c-b420-24b7b5dd51e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:04.817804 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7e725-ee8e-4e1c-b420-24b7b5dd51e6 None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:04.818526 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7e725-ee8e-4e1c-b420-24b7b5dd51e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:04.820063 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7e725-ee8e-4e1c-b420-24b7b5dd51e6 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:04.847734 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 31/61] 10.222.0.121 () {60 vars in 1137 bytes} [Thu Aug 4 10:18:04 2022] POST /identity/v3/users => generated 276 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:18:08.496698 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cee99-5f3d-4290-9dc7-da9f9c5c6bab None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:08.497626 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cee99-5f3d-4290-9dc7-da9f9c5c6bab None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:08.498063 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cee99-5f3d-4290-9dc7-da9f9c5c6bab None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:08.499626 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 31/62] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:08 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:08.505597 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67031d8b-5c72-437f-9734-a48aabbdb881 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:08.506609 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67031d8b-5c72-437f-9734-a48aabbdb881 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:08.507158 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67031d8b-5c72-437f-9734-a48aabbdb881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:08.537272 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-67031d8b-5c72-437f-9734-a48aabbdb881 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:08.538958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-67031d8b-5c72-437f-9734-a48aabbdb881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:08.539207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b05e1b-d5bd-4b8c-8d55-1023cc980827 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:08.539507 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b05e1b-d5bd-4b8c-8d55-1023cc980827 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:08.539790 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b05e1b-d5bd-4b8c-8d55-1023cc980827 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:08.542064 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 32/63] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:08 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:08.548472 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283ada7-8029-44dd-8e82-4813aafb48d0 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:08.549045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283ada7-8029-44dd-8e82-4813aafb48d0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:08.549476 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283ada7-8029-44dd-8e82-4813aafb48d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:08.579807 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0283ada7-8029-44dd-8e82-4813aafb48d0 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:08.581695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0283ada7-8029-44dd-8e82-4813aafb48d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:08.633601 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:08.641446 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 32/64] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:08 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:08.644540 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b174f-57ca-4bb7-a3dc-ad27e2f8115a None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:08.645169 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b174f-57ca-4bb7-a3dc-ad27e2f8115a None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:08.645641 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b174f-57ca-4bb7-a3dc-ad27e2f8115a None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:08.647357 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 33/65] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:08 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:08.656891 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21158c6-2579-4d5a-acc5-909d0f5394b6 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:08.657494 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21158c6-2579-4d5a-acc5-909d0f5394b6 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:08.657949 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21158c6-2579-4d5a-acc5-909d0f5394b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:08.681660 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-a21158c6-2579-4d5a-acc5-909d0f5394b6 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:08.683185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-a21158c6-2579-4d5a-acc5-909d0f5394b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:08.701522 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 33/66] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:08 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:18:08.708168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283ada7-8029-44dd-8e82-4813aafb48d0 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:08.767271 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:08.797539 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 34/67] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:08 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:08.799275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21158c6-2579-4d5a-acc5-909d0f5394b6 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:08.848903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283ada7-8029-44dd-8e82-4813aafb48d0 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:08.888491 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1580837-bb9b-4913-95ae-132740b7b95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:08.889458 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1580837-bb9b-4913-95ae-132740b7b95e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:08.889901 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1580837-bb9b-4913-95ae-132740b7b95e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:08.890328 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1580837-bb9b-4913-95ae-132740b7b95e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:08.903599 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1580837-bb9b-4913-95ae-132740b7b95e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:08.909080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-d1580837-bb9b-4913-95ae-132740b7b95e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:18:08.909080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:08.909080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:08.909080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:08.909080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:08.909080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:08.909080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:08.909080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:08.909080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:08.909080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:08.909080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:08.909080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:08.909080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:08.909080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:08.909080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:08.909080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:08.910159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:08.910159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:08.910159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:08.910159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:08.910159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:08.910159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:08.910159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:08.910159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:08.910159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:08.910159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:08.910159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:08.910159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:08.910159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:08.910159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:08.910159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:08.910159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:08.911129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:08.911129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:08.911129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:08.911129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:08.911129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:08.911129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:08.911129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:08.911129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:08.911129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:08.911129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:08.911129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:18:08.911129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:18:08.913644 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 34/68] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:18:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:08.915771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eeb892-6182-472a-b679-89aaeedef477 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:08.916100 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eeb892-6182-472a-b679-89aaeedef477 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:08.916390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eeb892-6182-472a-b679-89aaeedef477 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:08.917914 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 35/69] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:08 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:08.918913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eeb892-6182-472a-b679-89aaeedef477 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:08.933902 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21158c6-2579-4d5a-acc5-909d0f5394b6 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:08.964773 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:08.975130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e16a63-17e1-4e2a-8aad-790e5469ac85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:08.976010 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e16a63-17e1-4e2a-8aad-790e5469ac85 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:08.976309 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e16a63-17e1-4e2a-8aad-790e5469ac85 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:08.976592 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e16a63-17e1-4e2a-8aad-790e5469ac85 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:08.990303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e16a63-17e1-4e2a-8aad-790e5469ac85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:08.995436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-a5e16a63-17e1-4e2a-8aad-790e5469ac85 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:08.995436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:08.995436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:08.995436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:08.995436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:08.995436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:08.995436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:08.995436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:08.995436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:08.995436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:08.995436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:08.995436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:08.995436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:08.995436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:08.995436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:08.995436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:08.996400 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:08.996400 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:08.996400 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:08.996400 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:08.996400 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:08.996400 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:08.996400 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:08.996400 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:08.996400 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:08.996400 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:08.996400 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:08.996400 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:08.996400 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:08.996400 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:08.996400 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:08.996400 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:08.997206 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:08.997206 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:08.997206 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:08.997206 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:08.997206 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:08.997206 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:08.997206 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:08.997206 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:08.997206 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:08.997206 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:08.997206 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:08.997206 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:08.997206 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 35/70] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:08.998443 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fef50-ef80-4fa6-a488-316e4418a21f None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:08.998751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fef50-ef80-4fa6-a488-316e4418a21f None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:08.999024 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fef50-ef80-4fa6-a488-316e4418a21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:09.017482 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-313fef50-ef80-4fa6-a488-316e4418a21f None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:09.018627 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-313fef50-ef80-4fa6-a488-316e4418a21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:09.030645 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:09.035061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eeb892-6182-472a-b679-89aaeedef477 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:09.070043 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3be84e-bd0d-441a-91e4-464254af2378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:09.070714 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3be84e-bd0d-441a-91e4-464254af2378 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:09.070987 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3be84e-bd0d-441a-91e4-464254af2378 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:09.071271 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3be84e-bd0d-441a-91e4-464254af2378 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:09.081996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3be84e-bd0d-441a-91e4-464254af2378 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:09.087032 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-4a3be84e-bd0d-441a-91e4-464254af2378 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:18:09.087032 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:09.087032 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:09.087032 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:09.087032 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:09.087032 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:09.087032 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:09.087032 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:09.087032 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:09.087032 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:09.087032 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:09.087032 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:09.087032 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:09.087032 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:09.087032 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:09.087032 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:09.088150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:09.088150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:09.088150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:09.088150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:09.088150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:09.088150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:09.088150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:09.088150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:09.088150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:09.088150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:09.088150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:09.088150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:09.088150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:09.088150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:09.088150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:09.088150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:09.088968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:09.088968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:09.088968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:09.088968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:09.088968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:09.088968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:09.088968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:09.088968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:09.088968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:09.088968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:09.088968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:18:09.088968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:18:09.088968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 36/71] 10.222.0.121 () {56 vars in 1119 bytes} [Thu Aug 4 10:18:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:09.089621 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3be84e-bd0d-441a-91e4-464254af2378 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:09.095213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 36/72] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:08 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:09.095213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407f446-0260-4596-9959-f1fc55aad7b3 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:09.095213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407f446-0260-4596-9959-f1fc55aad7b3 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:09.095213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407f446-0260-4596-9959-f1fc55aad7b3 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:09.095213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 37/73] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:09 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:09.098144 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407f446-0260-4596-9959-f1fc55aad7b3 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:09.195908 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3be84e-bd0d-441a-91e4-464254af2378 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:09.207591 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407f446-0260-4596-9959-f1fc55aad7b3 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:09.225502 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:09.235007 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7b9ee-5dc5-46e5-bcb9-6dab3f8f6e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:09.235656 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7b9ee-5dc5-46e5-bcb9-6dab3f8f6e3d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:09.235966 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7b9ee-5dc5-46e5-bcb9-6dab3f8f6e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:09.236259 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7b9ee-5dc5-46e5-bcb9-6dab3f8f6e3d None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:09.236991 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7b9ee-5dc5-46e5-bcb9-6dab3f8f6e3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:09.237299 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7b9ee-5dc5-46e5-bcb9-6dab3f8f6e3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:09.238740 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7b9ee-5dc5-46e5-bcb9-6dab3f8f6e3d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:09.243631 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d6eb1-fbba-48ec-9d08-31792d3e8241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:09.244417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d6eb1-fbba-48ec-9d08-31792d3e8241 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:09.244698 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d6eb1-fbba-48ec-9d08-31792d3e8241 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:09.244971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d6eb1-fbba-48ec-9d08-31792d3e8241 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:09.245701 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d6eb1-fbba-48ec-9d08-31792d3e8241 None admin] RBAC: Adding query filter params (name=member) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:09.246006 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d6eb1-fbba-48ec-9d08-31792d3e8241 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:09.247397 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d6eb1-fbba-48ec-9d08-31792d3e8241 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:09.247564 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 37/74] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:09 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:09.248422 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7b9ee-5dc5-46e5-bcb9-6dab3f8f6e3d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:09.263655 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 38/75] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:09 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:09.269434 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375206c6-5776-4f7f-b0aa-2f60fe7f393f None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:09.269652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375206c6-5776-4f7f-b0aa-2f60fe7f393f None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:09.269936 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375206c6-5776-4f7f-b0aa-2f60fe7f393f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:09.275226 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:09.288790 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-375206c6-5776-4f7f-b0aa-2f60fe7f393f None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:09.290028 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-375206c6-5776-4f7f-b0aa-2f60fe7f393f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:09.350707 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7b9ee-5dc5-46e5-bcb9-6dab3f8f6e3d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:09.357651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 39/76] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:09 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:09.358540 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375206c6-5776-4f7f-b0aa-2f60fe7f393f None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:09.392285 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0856ddb-3cd7-49ed-9641-017a4d2c7628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:09.392928 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0856ddb-3cd7-49ed-9641-017a4d2c7628 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:09.393218 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0856ddb-3cd7-49ed-9641-017a4d2c7628 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:09.393500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0856ddb-3cd7-49ed-9641-017a4d2c7628 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:09.394242 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0856ddb-3cd7-49ed-9641-017a4d2c7628 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:09.394554 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0856ddb-3cd7-49ed-9641-017a4d2c7628 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:09.396009 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0856ddb-3cd7-49ed-9641-017a4d2c7628 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:09.404341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 38/77] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:18:09 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:09.405157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0856ddb-3cd7-49ed-9641-017a4d2c7628 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:09.463448 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375206c6-5776-4f7f-b0aa-2f60fe7f393f None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:09.474482 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:09.493246 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:09.500609 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd552b-bc73-4cb9-a70d-b751ad122586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:09.501285 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd552b-bc73-4cb9-a70d-b751ad122586 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:09.501573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd552b-bc73-4cb9-a70d-b751ad122586 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:09.501850 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd552b-bc73-4cb9-a70d-b751ad122586 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:09.509744 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0856ddb-3cd7-49ed-9641-017a4d2c7628 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:09.512323 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd552b-bc73-4cb9-a70d-b751ad122586 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:09.517461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-9cdd552b-bc73-4cb9-a70d-b751ad122586 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:09.517461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:09.517461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:09.517461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:09.517461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:09.517461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:09.517461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:09.517461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:09.517461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:09.517461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:09.517461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:09.517461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:09.517461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:09.517461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:09.517461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:09.517461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:09.518379 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:09.518379 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:09.518379 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:09.518379 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:09.518379 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:09.518379 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:09.518379 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:09.518379 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:09.518379 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:09.518379 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:09.518379 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:09.518379 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:09.518379 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:09.518379 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:09.518379 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:09.518379 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:09.519174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:09.519174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:09.519174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:09.519174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:09.519174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:09.519174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:09.519174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:09.519174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:09.519174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:09.519174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:09.519174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:09.519174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:09.519174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 40/78] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:09.520574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd552b-bc73-4cb9-a70d-b751ad122586 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:09.548321 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965fb4b1-182f-4a8f-84ea-8a6a9a2f56da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:09.548986 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fb4b1-182f-4a8f-84ea-8a6a9a2f56da None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:09.549265 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fb4b1-182f-4a8f-84ea-8a6a9a2f56da None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:09.549537 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fb4b1-182f-4a8f-84ea-8a6a9a2f56da None admin] PATH_INFO: `/v3/users/4d1eaa7a05e745a5817149d8a56dab58` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:09.550754 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965fb4b1-182f-4a8f-84ea-8a6a9a2f56da None admin] RBAC: Authorizing `identity:get_user(user_id=4d1eaa7a05e745a5817149d8a56dab58)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:09.552281 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965fb4b1-182f-4a8f-84ea-8a6a9a2f56da None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:09.553980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 39/79] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:09 2022] GET /identity/v3/users/4d1eaa7a05e745a5817149d8a56dab58 => generated 248 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:09.554921 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965fb4b1-182f-4a8f-84ea-8a6a9a2f56da None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:09.642193 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd552b-bc73-4cb9-a70d-b751ad122586 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:09.666547 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965fb4b1-182f-4a8f-84ea-8a6a9a2f56da None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:09.681290 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4f8c8-1167-4824-8db8-45bdcc6f4374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:09.681967 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4f8c8-1167-4824-8db8-45bdcc6f4374 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:09.682256 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4f8c8-1167-4824-8db8-45bdcc6f4374 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:09.682536 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4f8c8-1167-4824-8db8-45bdcc6f4374 None admin] PATH_INFO: `/v3/users/5b20a8c5e8314c8794af566d108815a8` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:09.698983 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4f8c8-1167-4824-8db8-45bdcc6f4374 None admin] RBAC: Authorizing `identity:get_user(user_id=5b20a8c5e8314c8794af566d108815a8)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:09.700534 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4f8c8-1167-4824-8db8-45bdcc6f4374 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:09.702628 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 41/80] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:09 2022] GET /identity/v3/users/5b20a8c5e8314c8794af566d108815a8 => generated 276 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:09.703456 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4f8c8-1167-4824-8db8-45bdcc6f4374 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:09.711346 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6013ea6-de5e-4a75-a6e3-41d4fe4c1984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:09.712058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6013ea6-de5e-4a75-a6e3-41d4fe4c1984 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:09.712347 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6013ea6-de5e-4a75-a6e3-41d4fe4c1984 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:09.712627 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6013ea6-de5e-4a75-a6e3-41d4fe4c1984 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:09.714027 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6013ea6-de5e-4a75-a6e3-41d4fe4c1984 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:09.716149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6013ea6-de5e-4a75-a6e3-41d4fe4c1984 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:09.718868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 40/81] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:18:09 2022] GET /identity/v3/domains/default => generated 200 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:09.727151 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6013ea6-de5e-4a75-a6e3-41d4fe4c1984 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:09.744900 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:09.746567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:09.821713 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4f8c8-1167-4824-8db8-45bdcc6f4374 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:09.829810 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6013ea6-de5e-4a75-a6e3-41d4fe4c1984 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:09.857228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe731200-35f8-4fb6-beff-01e68c2c8337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:09.858187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe731200-35f8-4fb6-beff-01e68c2c8337 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:09.858475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe731200-35f8-4fb6-beff-01e68c2c8337 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:09.858758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe731200-35f8-4fb6-beff-01e68c2c8337 None admin] PATH_INFO: `/v3/users/5b20a8c5e8314c8794af566d108815a8` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:09.859965 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe731200-35f8-4fb6-beff-01e68c2c8337 None admin] RBAC: Authorizing `identity:get_user(user_id=5b20a8c5e8314c8794af566d108815a8)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:09.861488 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe731200-35f8-4fb6-beff-01e68c2c8337 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:09.863167 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 42/82] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:09 2022] GET /identity/v3/users/5b20a8c5e8314c8794af566d108815a8 => generated 276 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:09.864307 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe731200-35f8-4fb6-beff-01e68c2c8337 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:09.867056 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05371b-d0b9-4db2-8a73-4ea4c7f21887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:09.867707 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05371b-d0b9-4db2-8a73-4ea4c7f21887 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:09.868033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05371b-d0b9-4db2-8a73-4ea4c7f21887 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:09.868318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05371b-d0b9-4db2-8a73-4ea4c7f21887 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:09.869052 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05371b-d0b9-4db2-8a73-4ea4c7f21887 None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:09.869355 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05371b-d0b9-4db2-8a73-4ea4c7f21887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:09.870757 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05371b-d0b9-4db2-8a73-4ea4c7f21887 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:09.880526 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 41/83] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:09 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:09.884907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05371b-d0b9-4db2-8a73-4ea4c7f21887 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:09.969220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:09.975833 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe731200-35f8-4fb6-beff-01e68c2c8337 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:09.976671 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:09.998225 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05371b-d0b9-4db2-8a73-4ea4c7f21887 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:10.012190 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f81b33-90b1-41a8-9d15-7098a88400c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:10.013024 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f81b33-90b1-41a8-9d15-7098a88400c8 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:10.013312 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f81b33-90b1-41a8-9d15-7098a88400c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:10.013584 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f81b33-90b1-41a8-9d15-7098a88400c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:10.032223 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62d55f-233b-4450-b070-dd5b6fb595d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:10.032863 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62d55f-233b-4450-b070-dd5b6fb595d4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:10.033150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62d55f-233b-4450-b070-dd5b6fb595d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:10.033455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62d55f-233b-4450-b070-dd5b6fb595d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:10.083846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f81b33-90b1-41a8-9d15-7098a88400c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:10.085351 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f81b33-90b1-41a8-9d15-7098a88400c8 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:10.102430 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b62d55f-233b-4450-b070-dd5b6fb595d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:10.104020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b62d55f-233b-4450-b070-dd5b6fb595d4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:10.187594 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:10.191902 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:10.196168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 43/84] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:18:09 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:18:10.197047 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f81b33-90b1-41a8-9d15-7098a88400c8 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:10.213854 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 42/85] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:18:09 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:18:10.214993 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62d55f-233b-4450-b070-dd5b6fb595d4 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:10.305667 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f81b33-90b1-41a8-9d15-7098a88400c8 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:10.322591 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62d55f-233b-4450-b070-dd5b6fb595d4 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:10.341193 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a1c05-eadc-4e5e-8c7a-5521109ed48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:10.341748 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a1c05-eadc-4e5e-8c7a-5521109ed48a None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:10.342035 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a1c05-eadc-4e5e-8c7a-5521109ed48a None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:10.342315 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a1c05-eadc-4e5e-8c7a-5521109ed48a None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:10.343013 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3a1c05-eadc-4e5e-8c7a-5521109ed48a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:10.344542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3a1c05-eadc-4e5e-8c7a-5521109ed48a None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:10.359096 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22fb6b9-9f92-46af-aa08-f28d09d4ed19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:10.359747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22fb6b9-9f92-46af-aa08-f28d09d4ed19 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:10.360036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22fb6b9-9f92-46af-aa08-f28d09d4ed19 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:10.360318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22fb6b9-9f92-46af-aa08-f28d09d4ed19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:10.366312 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 44/86] 10.222.0.121 () {60 vars in 1137 bytes} [Thu Aug 4 10:18:10 2022] POST /identity/v3/users => generated 290 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:18:10.367205 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a1c05-eadc-4e5e-8c7a-5521109ed48a None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:10.430609 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:10.431735 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:10.455930 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22fb6b9-9f92-46af-aa08-f28d09d4ed19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:10.457530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22fb6b9-9f92-46af-aa08-f28d09d4ed19 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:10.497431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a1c05-eadc-4e5e-8c7a-5521109ed48a None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:10.546831 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef2bca-0774-482e-8667-3359c340e61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:10.546831 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef2bca-0774-482e-8667-3359c340e61a None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:10.547636 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef2bca-0774-482e-8667-3359c340e61a None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:10.547636 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef2bca-0774-482e-8667-3359c340e61a None admin] PATH_INFO: `/v3/users/5b20a8c5e8314c8794af566d108815a8` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:10.551890 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef2bca-0774-482e-8667-3359c340e61a None admin] RBAC: Authorizing `identity:get_user(user_id=5b20a8c5e8314c8794af566d108815a8)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:10.554584 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef2bca-0774-482e-8667-3359c340e61a None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:10.558247 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 45/87] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:10 2022] GET /identity/v3/users/5b20a8c5e8314c8794af566d108815a8 => generated 276 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:10.559006 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef2bca-0774-482e-8667-3359c340e61a None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:10.589499 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 43/88] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:18:10 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:18:10.590341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22fb6b9-9f92-46af-aa08-f28d09d4ed19 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:10.702747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:10.704888 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef2bca-0774-482e-8667-3359c340e61a None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:10.724586 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22fb6b9-9f92-46af-aa08-f28d09d4ed19 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:10.728579 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:10.746337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe4539-895c-405d-926c-8201b96c9a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:10.747036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe4539-895c-405d-926c-8201b96c9a22 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:10.747326 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe4539-895c-405d-926c-8201b96c9a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:10.747608 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe4539-895c-405d-926c-8201b96c9a22 None admin] PATH_INFO: `/v3/projects/6d7a7a0b84384f14849056212228badb` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:10.749002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe4539-895c-405d-926c-8201b96c9a22 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7a7a0b84384f14849056212228badb)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:10.750575 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe4539-895c-405d-926c-8201b96c9a22 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:10.753042 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 46/89] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:10 2022] GET /identity/v3/projects/6d7a7a0b84384f14849056212228badb => generated 299 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:10.753906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe4539-895c-405d-926c-8201b96c9a22 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:10.764630 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e8c2e-7142-4914-bac0-914488819ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:10.765296 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e8c2e-7142-4914-bac0-914488819ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:10.765587 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e8c2e-7142-4914-bac0-914488819ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:10.765865 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e8c2e-7142-4914-bac0-914488819ca6 None admin] PATH_INFO: `/v3/projects/6d7a7a0b84384f14849056212228badb` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:10.767082 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4e8c2e-7142-4914-bac0-914488819ca6 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7a7a0b84384f14849056212228badb)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:10.768612 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4e8c2e-7142-4914-bac0-914488819ca6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:10.770578 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 44/90] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:10 2022] GET /identity/v3/projects/6d7a7a0b84384f14849056212228badb => generated 299 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:10.771132 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e8c2e-7142-4914-bac0-914488819ca6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:10.890195 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e8c2e-7142-4914-bac0-914488819ca6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:10.900948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe4539-895c-405d-926c-8201b96c9a22 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:10.941039 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b7c33-0321-4c13-8522-e54eba29fa5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:10.942427 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b7c33-0321-4c13-8522-e54eba29fa5c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:10.943388 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b7c33-0321-4c13-8522-e54eba29fa5c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:10.944151 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b7c33-0321-4c13-8522-e54eba29fa5c None admin] PATH_INFO: `/v3/projects/6d7a7a0b84384f14849056212228badb` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:10.946093 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b7c33-0321-4c13-8522-e54eba29fa5c None admin] RBAC: Authorizing `identity:get_project(project_id=6d7a7a0b84384f14849056212228badb)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:10.947900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184ed36-3c7e-4793-b1e5-13a7ead4c01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:10.948358 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b7c33-0321-4c13-8522-e54eba29fa5c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:10.948591 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184ed36-3c7e-4793-b1e5-13a7ead4c01a None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:10.948881 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184ed36-3c7e-4793-b1e5-13a7ead4c01a None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:10.949165 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184ed36-3c7e-4793-b1e5-13a7ead4c01a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:10.951036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 45/91] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:10 2022] GET /identity/v3/projects/6d7a7a0b84384f14849056212228badb => generated 299 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:10.952744 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b7c33-0321-4c13-8522-e54eba29fa5c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:10.979455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:11.022127 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:11.036851 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184ed36-3c7e-4793-b1e5-13a7ead4c01a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:11.038433 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184ed36-3c7e-4793-b1e5-13a7ead4c01a None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:11.108242 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b7c33-0321-4c13-8522-e54eba29fa5c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:11.154650 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61912a-cda7-49f4-a127-d8ab1558e6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:11.157507 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61912a-cda7-49f4-a127-d8ab1558e6c6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:11.157990 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61912a-cda7-49f4-a127-d8ab1558e6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:11.158447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61912a-cda7-49f4-a127-d8ab1558e6c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:11.160078 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61912a-cda7-49f4-a127-d8ab1558e6c6 None admin] RBAC: Adding query filter params (role.id=7342106a255b4bce8f150726602a8b95, scope.project.id=6d7a7a0b84384f14849056212228badb, user.id=4d1eaa7a05e745a5817149d8a56dab58) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:11.160609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61912a-cda7-49f4-a127-d8ab1558e6c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:11.163689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61912a-cda7-49f4-a127-d8ab1558e6c6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:11.171777 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 47/92] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:18:10 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:18:11.173149 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184ed36-3c7e-4793-b1e5-13a7ead4c01a None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:11.178609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 46/93] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:18:10 2022] GET /identity/v3/role_assignments?user.id=4d1eaa7a05e745a5817149d8a56dab58&scope.project.id=6d7a7a0b84384f14849056212228badb&role.id=7342106a255b4bce8f150726602a8b95 => generated 260 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:11.180183 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61912a-cda7-49f4-a127-d8ab1558e6c6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:11.300337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:11.321746 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184ed36-3c7e-4793-b1e5-13a7ead4c01a None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:11.335018 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:11.371117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61912a-cda7-49f4-a127-d8ab1558e6c6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:11.373209 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6758c-0420-48a4-986d-ef9673029f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:11.374564 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6758c-0420-48a4-986d-ef9673029f4b None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:11.375100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6758c-0420-48a4-986d-ef9673029f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:11.376833 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6758c-0420-48a4-986d-ef9673029f4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:11.377966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a6758c-0420-48a4-986d-ef9673029f4b None admin] RBAC: Adding query filter params (role.id=1b988376d12749bd8d653baa89493c5c, scope.project.id=6d7a7a0b84384f14849056212228badb, user.id=5b20a8c5e8314c8794af566d108815a8) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:11.378504 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a6758c-0420-48a4-986d-ef9673029f4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:11.380304 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a6758c-0420-48a4-986d-ef9673029f4b None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:11.390001 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 48/94] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:18:11 2022] GET /identity/v3/role_assignments?user.id=5b20a8c5e8314c8794af566d108815a8&scope.project.id=6d7a7a0b84384f14849056212228badb&role.id=1b988376d12749bd8d653baa89493c5c => generated 260 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:11.391455 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6758c-0420-48a4-986d-ef9673029f4b None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:11.427180 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01702e-873a-440a-8914-4d817db33354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:11.427619 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01702e-873a-440a-8914-4d817db33354 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:11.427902 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01702e-873a-440a-8914-4d817db33354 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:11.428241 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01702e-873a-440a-8914-4d817db33354 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:11.429401 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01702e-873a-440a-8914-4d817db33354 None admin] RBAC: Adding query filter params (role.id=89b847fca3574b9083287279aee0f773, scope.project.id=6d7a7a0b84384f14849056212228badb, user.id=5b20a8c5e8314c8794af566d108815a8) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:11.429846 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01702e-873a-440a-8914-4d817db33354 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:11.431308 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01702e-873a-440a-8914-4d817db33354 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:11.437969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 47/95] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:18:11 2022] GET /identity/v3/role_assignments?user.id=5b20a8c5e8314c8794af566d108815a8&scope.project.id=6d7a7a0b84384f14849056212228badb&role.id=89b847fca3574b9083287279aee0f773 => generated 260 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:11.547383 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6758c-0420-48a4-986d-ef9673029f4b None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:11.613751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a1c4c-ca65-413e-ba15-b1ac5cdd41d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:11.614808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a1c4c-ca65-413e-ba15-b1ac5cdd41d2 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:11.615100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a1c4c-ca65-413e-ba15-b1ac5cdd41d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:11.615384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a1c4c-ca65-413e-ba15-b1ac5cdd41d2 None admin] PATH_INFO: `/v3/projects/febd81c48dc747079c83424c156e5e27` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:11.621222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a1c4c-ca65-413e-ba15-b1ac5cdd41d2 None admin] RBAC: Authorizing `identity:get_project(project_id=febd81c48dc747079c83424c156e5e27)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:11.622844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a1c4c-ca65-413e-ba15-b1ac5cdd41d2 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:11.624878 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 49/96] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:11 2022] GET /identity/v3/projects/febd81c48dc747079c83424c156e5e27 => generated 313 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:11.631932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01702e-873a-440a-8914-4d817db33354 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:11.789852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01702e-873a-440a-8914-4d817db33354 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:11.833260 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a3c035-75d4-4e5e-9fc0-6e5a35ca4683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:11.834367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3c035-75d4-4e5e-9fc0-6e5a35ca4683 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:11.834903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3c035-75d4-4e5e-9fc0-6e5a35ca4683 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:11.835440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3c035-75d4-4e5e-9fc0-6e5a35ca4683 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:11.836589 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a3c035-75d4-4e5e-9fc0-6e5a35ca4683 None admin] RBAC: Adding query filter params (role.id=1b988376d12749bd8d653baa89493c5c, scope.project.id=febd81c48dc747079c83424c156e5e27, user.id=5b20a8c5e8314c8794af566d108815a8) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:11.837145 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a3c035-75d4-4e5e-9fc0-6e5a35ca4683 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:11.838896 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a3c035-75d4-4e5e-9fc0-6e5a35ca4683 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:11.846143 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 48/97] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:18:11 2022] GET /identity/v3/role_assignments?user.id=5b20a8c5e8314c8794af566d108815a8&scope.project.id=febd81c48dc747079c83424c156e5e27&role.id=1b988376d12749bd8d653baa89493c5c => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:12.803736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586d453-9bb3-41e6-b2cd-2ff68bf05486 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:12.804519 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586d453-9bb3-41e6-b2cd-2ff68bf05486 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:12.804519 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586d453-9bb3-41e6-b2cd-2ff68bf05486 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:12.805908 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 50/98] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:12.817583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b85ab-c4e3-4b03-a30c-66320717012a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:12.818224 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b85ab-c4e3-4b03-a30c-66320717012a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:12.818224 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b85ab-c4e3-4b03-a30c-66320717012a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:12.843323 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ae5b85ab-c4e3-4b03-a30c-66320717012a None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:12.844607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ae5b85ab-c4e3-4b03-a30c-66320717012a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:12.850538 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:12.936696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 49/99] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:12 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:12.972042 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586d453-9bb3-41e6-b2cd-2ff68bf05486 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:13.005375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:13.120369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586d453-9bb3-41e6-b2cd-2ff68bf05486 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:13.150767 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54f255-350a-493e-a285-3e634a8627e5 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:13.151048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54f255-350a-493e-a285-3e634a8627e5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:13.151339 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54f255-350a-493e-a285-3e634a8627e5 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:13.153045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 50/100] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:13.159558 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c5315-a904-4703-94b8-22cc28748c9d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:13.159828 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c5315-a904-4703-94b8-22cc28748c9d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:13.160178 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c5315-a904-4703-94b8-22cc28748c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:13.193226 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f15c5315-a904-4703-94b8-22cc28748c9d None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:13.194517 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f15c5315-a904-4703-94b8-22cc28748c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:13.199221 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c9fac-a773-4141-95fc-c19760a49af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:13.199922 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c9fac-a773-4141-95fc-c19760a49af9 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:13.200251 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c9fac-a773-4141-95fc-c19760a49af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:13.200535 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c9fac-a773-4141-95fc-c19760a49af9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:13.223253 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c9fac-a773-4141-95fc-c19760a49af9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:13.229195 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-9b1c9fac-a773-4141-95fc-c19760a49af9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:18:13.229195 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:13.229195 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:13.229195 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:13.229195 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:13.229195 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:13.229195 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:13.229195 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:13.229195 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:13.229195 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:13.229195 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:13.229195 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:13.229195 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:13.229195 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:13.229195 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:13.229195 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:13.230235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:13.230235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:13.230235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:13.230235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:13.230235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:13.230235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:13.230235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:13.230235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:13.230235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:13.230235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:13.230235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:13.230235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:13.230235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:13.230235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:13.230235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:13.230235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:13.231064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:13.231064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:13.231064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:13.231064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:13.231064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:13.231064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:13.231064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:13.231064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:13.231064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:13.231064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:13.231064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:18:13.231064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:13.231064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 51/101] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:13.239227 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c9fac-a773-4141-95fc-c19760a49af9 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:13.401909 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 51/102] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:13 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:18:13.446892 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c5315-a904-4703-94b8-22cc28748c9d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:13.448148 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:13.506910 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c9fac-a773-4141-95fc-c19760a49af9 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:13.566239 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8694225-6092-41e1-aab7-8e1cd7a0a9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:13.566911 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8694225-6092-41e1-aab7-8e1cd7a0a9be None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:13.567207 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8694225-6092-41e1-aab7-8e1cd7a0a9be None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:13.567495 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8694225-6092-41e1-aab7-8e1cd7a0a9be None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:13.568289 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8694225-6092-41e1-aab7-8e1cd7a0a9be None admin] RBAC: Adding query filter params (name=reader) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:13.568606 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8694225-6092-41e1-aab7-8e1cd7a0a9be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:13.570145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8694225-6092-41e1-aab7-8e1cd7a0a9be None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:13.587510 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 52/103] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:13 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:13.594707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8694225-6092-41e1-aab7-8e1cd7a0a9be None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:13.650143 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:13.763899 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c5315-a904-4703-94b8-22cc28748c9d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:13.772369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8694225-6092-41e1-aab7-8e1cd7a0a9be None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:13.840762 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b66bad2-b042-4066-8107-8beb1d512f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:13.841695 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66bad2-b042-4066-8107-8beb1d512f61 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:13.841831 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66bad2-b042-4066-8107-8beb1d512f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:13.842176 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66bad2-b042-4066-8107-8beb1d512f61 None admin] PATH_INFO: `/v3/users/d7693ebbe28d47628c3a1ec1cdff6144` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:13.860144 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b068a-5609-43b9-86ee-b87066483311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:13.868528 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b068a-5609-43b9-86ee-b87066483311 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:13.868940 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b66bad2-b042-4066-8107-8beb1d512f61 None admin] RBAC: Authorizing `identity:get_user(user_id=d7693ebbe28d47628c3a1ec1cdff6144)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:13.869341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b068a-5609-43b9-86ee-b87066483311 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:13.869836 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b068a-5609-43b9-86ee-b87066483311 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:13.872148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b068a-5609-43b9-86ee-b87066483311 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:13.873903 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:13.874658 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b66bad2-b042-4066-8107-8beb1d512f61 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:13.877108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 53/104] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:13 2022] GET /identity/v3/users/d7693ebbe28d47628c3a1ec1cdff6144 => generated 290 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:13.878771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b068a-5609-43b9-86ee-b87066483311 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:13.883693 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 52/105] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:18:13 2022] GET /identity/v3/domains/default => generated 200 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:18:13.884160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b66bad2-b042-4066-8107-8beb1d512f61 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:13.892684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b068a-5609-43b9-86ee-b87066483311 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:14.027445 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b068a-5609-43b9-86ee-b87066483311 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:14.065673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bb3d1-7d7f-4951-8ba2-a7e4dc9fc5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:14.066434 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bb3d1-7d7f-4951-8ba2-a7e4dc9fc5e1 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:14.066730 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bb3d1-7d7f-4951-8ba2-a7e4dc9fc5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:14.067014 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bb3d1-7d7f-4951-8ba2-a7e4dc9fc5e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:14.067750 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bb3d1-7d7f-4951-8ba2-a7e4dc9fc5e1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:14.069289 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bb3d1-7d7f-4951-8ba2-a7e4dc9fc5e1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:14.080264 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:14.116979 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 53/106] 10.222.0.121 () {60 vars in 1146 bytes} [Thu Aug 4 10:18:13 2022] POST /identity/v3/projects => generated 303 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:18:14.131413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b66bad2-b042-4066-8107-8beb1d512f61 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:14.207986 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ade34-b5fb-4d02-a064-a17ddf4b4274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:14.209085 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ade34-b5fb-4d02-a064-a17ddf4b4274 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:14.209584 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ade34-b5fb-4d02-a064-a17ddf4b4274 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:14.210086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ade34-b5fb-4d02-a064-a17ddf4b4274 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:14.372411 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:14.380367 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ade34-b5fb-4d02-a064-a17ddf4b4274 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:14.384671 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ade34-b5fb-4d02-a064-a17ddf4b4274 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:14.593656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 54/107] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:18:13 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 710 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 10:18:14.601498 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bb3d1-7d7f-4951-8ba2-a7e4dc9fc5e1 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:14.881133 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bb3d1-7d7f-4951-8ba2-a7e4dc9fc5e1 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:14.965602 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c7922-bf2a-42a7-8959-e92e5d851395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:14.966786 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c7922-bf2a-42a7-8959-e92e5d851395 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:14.967302 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c7922-bf2a-42a7-8959-e92e5d851395 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:14.967816 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c7922-bf2a-42a7-8959-e92e5d851395 None admin] PATH_INFO: `/v3/projects/6d7a7a0b84384f14849056212228badb` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:14.970343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c7922-bf2a-42a7-8959-e92e5d851395 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7a7a0b84384f14849056212228badb)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:14.973386 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c7922-bf2a-42a7-8959-e92e5d851395 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:14.976778 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 54/108] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:14 2022] GET /identity/v3/projects/6d7a7a0b84384f14849056212228badb => generated 299 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:14.987198 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ade34-b5fb-4d02-a064-a17ddf4b4274 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:15.003244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c728f-a795-4940-8dbf-a433d96c81b3 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:15.003769 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c728f-a795-4940-8dbf-a433d96c81b3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:15.004231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c728f-a795-4940-8dbf-a433d96c81b3 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:15.007425 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 55/109] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:15 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:15.016771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c67ab-00b5-435f-96ad-d5c8af4e36c9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:15.017230 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c67ab-00b5-435f-96ad-d5c8af4e36c9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:15.017726 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c67ab-00b5-435f-96ad-d5c8af4e36c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:15.085627 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0b1c67ab-00b5-435f-96ad-d5c8af4e36c9 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:15.087343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0b1c67ab-00b5-435f-96ad-d5c8af4e36c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:15.268613 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:15.301765 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:15.306657 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ade34-b5fb-4d02-a064-a17ddf4b4274 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:15.310132 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 56/110] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:15 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:15.354052 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34016ddd-7b0b-4e22-b61f-0bfde2e8545b None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:15.354445 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34016ddd-7b0b-4e22-b61f-0bfde2e8545b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:15.354617 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34016ddd-7b0b-4e22-b61f-0bfde2e8545b None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:15.356933 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 57/111] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:15 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:15.363101 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f3c57-cdeb-4c06-bc43-415e2c7ab6fb None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:15.363576 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f3c57-cdeb-4c06-bc43-415e2c7ab6fb None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:15.364387 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f3c57-cdeb-4c06-bc43-415e2c7ab6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:15.400811 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-fc9f3c57-cdeb-4c06-bc43-415e2c7ab6fb None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:15.401341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03fc27-be3a-455c-afc3-d15901569bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:15.402246 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03fc27-be3a-455c-afc3-d15901569bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:15.402678 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03fc27-be3a-455c-afc3-d15901569bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:15.402960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03fc27-be3a-455c-afc3-d15901569bb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:15.403831 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03fc27-be3a-455c-afc3-d15901569bb9 None admin] RBAC: Adding query filter params (role.id=c77f1c570e8640a3b2c836a401b91889, scope.project.id=6d7a7a0b84384f14849056212228badb, user.id=d7693ebbe28d47628c3a1ec1cdff6144) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:15.404190 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03fc27-be3a-455c-afc3-d15901569bb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:15.404496 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fc9f3c57-cdeb-4c06-bc43-415e2c7ab6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:15.405701 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03fc27-be3a-455c-afc3-d15901569bb9 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:15.414776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 55/112] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:18:14 2022] GET /identity/v3/role_assignments?user.id=d7693ebbe28d47628c3a1ec1cdff6144&scope.project.id=6d7a7a0b84384f14849056212228badb&role.id=c77f1c570e8640a3b2c836a401b91889 => generated 260 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:15.417047 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4707d3-0a69-4c2c-965a-9cd9d26b54c9 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:15.417586 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4707d3-0a69-4c2c-965a-9cd9d26b54c9 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:15.418020 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4707d3-0a69-4c2c-965a-9cd9d26b54c9 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:15.420665 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 56/113] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:15 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:15.424008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4707d3-0a69-4c2c-965a-9cd9d26b54c9 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:15.558571 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 58/114] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:15 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:18:15.563104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d6a35-c656-4dbe-b3fc-8a5789c3103e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:15.564039 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d6a35-c656-4dbe-b3fc-8a5789c3103e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:15.564635 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d6a35-c656-4dbe-b3fc-8a5789c3103e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:15.622084 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c16d6a35-c656-4dbe-b3fc-8a5789c3103e None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:15.628916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c16d6a35-c656-4dbe-b3fc-8a5789c3103e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:15.688308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4707d3-0a69-4c2c-965a-9cd9d26b54c9 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:15.728250 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 59/115] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:15 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:15.729851 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d6a35-c656-4dbe-b3fc-8a5789c3103e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:15.754114 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62eb97-895c-4f67-bc52-62b8f32e3e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:15.757031 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62eb97-895c-4f67-bc52-62b8f32e3e86 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:15.757031 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62eb97-895c-4f67-bc52-62b8f32e3e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:15.757031 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62eb97-895c-4f67-bc52-62b8f32e3e86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:15.779745 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62eb97-895c-4f67-bc52-62b8f32e3e86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:15.786415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-be62eb97-895c-4f67-bc52-62b8f32e3e86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:15.786415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:15.786415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:15.786415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:15.786415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:15.786415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:15.786415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:15.786415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:15.786415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:15.786415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:15.786415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:15.786415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:15.786415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:15.786415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:15.786415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:15.786415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:15.787434 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:15.787434 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:15.787434 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:15.787434 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:15.787434 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:15.787434 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:15.787434 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:15.787434 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:15.787434 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:15.787434 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:15.787434 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:15.787434 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:15.787434 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:15.787434 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:15.787434 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:15.787434 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:15.788345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:15.788345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:15.788345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:15.788345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:15.788345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:15.788345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:15.788345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:15.788345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:15.788345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:15.788345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:15.788345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:15.788345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:15.791071 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 57/116] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:15.792785 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af507f5-1cb7-4a41-bfd4-50d571a8c857 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:15.793375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af507f5-1cb7-4a41-bfd4-50d571a8c857 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:15.793876 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af507f5-1cb7-4a41-bfd4-50d571a8c857 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:15.795629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 58/117] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:15 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:15.801093 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af507f5-1cb7-4a41-bfd4-50d571a8c857 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:15.821298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:15.841348 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:15.887991 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d6a35-c656-4dbe-b3fc-8a5789c3103e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:15.932794 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec15edc-2f1e-4cbb-a0a2-60b44ad6e8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:15.933495 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec15edc-2f1e-4cbb-a0a2-60b44ad6e8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:15.933782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec15edc-2f1e-4cbb-a0a2-60b44ad6e8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:15.934062 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec15edc-2f1e-4cbb-a0a2-60b44ad6e8c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:15.948775 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec15edc-2f1e-4cbb-a0a2-60b44ad6e8c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:15.952998 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af507f5-1cb7-4a41-bfd4-50d571a8c857 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:15.953923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-5ec15edc-2f1e-4cbb-a0a2-60b44ad6e8c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:15.953923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:15.953923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:15.953923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:15.953923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:15.953923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:15.953923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:15.953923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:15.953923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:15.953923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:15.953923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:15.953923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:15.953923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:15.953923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:15.953923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:15.953923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:15.954829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:15.954829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:15.954829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:15.954829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:15.954829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:15.954829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:15.954829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:15.954829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:15.954829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:15.954829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:15.954829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:15.954829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:15.954829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:15.954829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:15.954829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:15.954829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:15.955625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:15.955625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:15.955625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:15.955625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:15.955625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:15.955625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:15.955625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:15.955625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:15.955625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:15.955625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:15.955625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:15.955625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:18:15.955625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 60/118] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:15.961150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec15edc-2f1e-4cbb-a0a2-60b44ad6e8c2 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:16.009558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f75df-6980-4ab9-8eb9-eee3e3c84b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:16.011189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f75df-6980-4ab9-8eb9-eee3e3c84b8b None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:16.011715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f75df-6980-4ab9-8eb9-eee3e3c84b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:16.012257 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f75df-6980-4ab9-8eb9-eee3e3c84b8b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:16.028111 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f75df-6980-4ab9-8eb9-eee3e3c84b8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:16.036134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-a58f75df-6980-4ab9-8eb9-eee3e3c84b8b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:18:16.036134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:16.036134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:16.036134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:16.036134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:16.036134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:16.036134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:16.036134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:16.036134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:16.036134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:16.036134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:16.036134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:16.036134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:16.036134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:16.036134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:16.036134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:16.037125 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:16.037125 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:16.037125 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:16.037125 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:16.037125 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:16.037125 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:16.037125 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:16.037125 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:16.037125 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:16.037125 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:16.037125 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:16.037125 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:16.037125 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:16.037125 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:16.037125 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:16.037125 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:16.037926 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:16.037926 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:16.037926 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:16.037926 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:16.037926 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:16.037926 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:16.037926 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:16.037926 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:16.037926 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:16.037926 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:16.037926 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:18:16.037926 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:16.049791 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 59/119] 10.222.0.121 () {56 vars in 1119 bytes} [Thu Aug 4 10:18:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:16.049791 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9be61-98ab-4074-a72b-7ffd580736c0 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:16.049791 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9be61-98ab-4074-a72b-7ffd580736c0 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:16.049791 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9be61-98ab-4074-a72b-7ffd580736c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:16.069877 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-3bd9be61-98ab-4074-a72b-7ffd580736c0 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:16.072229 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-3bd9be61-98ab-4074-a72b-7ffd580736c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:16.101485 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec15edc-2f1e-4cbb-a0a2-60b44ad6e8c2 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:16.114367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:16.166324 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bb48b-3985-42f3-b0c1-6ab489eb1212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:16.167366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bb48b-3985-42f3-b0c1-6ab489eb1212 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:16.168109 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bb48b-3985-42f3-b0c1-6ab489eb1212 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:16.168718 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bb48b-3985-42f3-b0c1-6ab489eb1212 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:16.170263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bb48b-3985-42f3-b0c1-6ab489eb1212 None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:16.170808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bb48b-3985-42f3-b0c1-6ab489eb1212 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:16.174214 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bb48b-3985-42f3-b0c1-6ab489eb1212 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:16.174950 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 60/120] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:16 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:16.183277 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9be61-98ab-4074-a72b-7ffd580736c0 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:16.208927 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 61/121] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:15 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:16.215982 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bb48b-3985-42f3-b0c1-6ab489eb1212 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:16.223695 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:16.347572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9be61-98ab-4074-a72b-7ffd580736c0 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:16.392661 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86668494-ac6c-4271-9661-11681b04eb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:16.393627 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86668494-ac6c-4271-9661-11681b04eb13 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:16.393973 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86668494-ac6c-4271-9661-11681b04eb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:16.394394 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86668494-ac6c-4271-9661-11681b04eb13 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:16.395215 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86668494-ac6c-4271-9661-11681b04eb13 None admin] RBAC: Adding query filter params (name=member) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:16.395534 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86668494-ac6c-4271-9661-11681b04eb13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:16.397129 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86668494-ac6c-4271-9661-11681b04eb13 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:16.409752 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 61/122] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:16 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:16.416647 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86668494-ac6c-4271-9661-11681b04eb13 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:16.425264 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bb48b-3985-42f3-b0c1-6ab489eb1212 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:16.501792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d0fc9-9035-4f85-9f67-bf79860658a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:16.503697 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d0fc9-9035-4f85-9f67-bf79860658a5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:16.504604 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d0fc9-9035-4f85-9f67-bf79860658a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:16.505378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d0fc9-9035-4f85-9f67-bf79860658a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:16.507172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d0fc9-9035-4f85-9f67-bf79860658a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:16.508186 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d0fc9-9035-4f85-9f67-bf79860658a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:16.511308 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d0fc9-9035-4f85-9f67-bf79860658a5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:16.527518 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:16.533036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 62/123] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:18:16 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:16.534976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d0fc9-9035-4f85-9f67-bf79860658a5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:16.566145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86668494-ac6c-4271-9661-11681b04eb13 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:16.605849 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:16.618023 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8a5b0-94db-4477-9d96-47650eacc676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:16.619737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8a5b0-94db-4477-9d96-47650eacc676 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:16.621435 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8a5b0-94db-4477-9d96-47650eacc676 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:16.622214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8a5b0-94db-4477-9d96-47650eacc676 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:16.651286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8a5b0-94db-4477-9d96-47650eacc676 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:16.666524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-45e8a5b0-94db-4477-9d96-47650eacc676 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:18:16.666524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:16.666524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:16.666524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:16.666524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:16.666524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:16.666524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:16.666524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:16.666524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:16.666524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:16.666524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:16.666524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:16.666524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:16.666524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:16.666524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:16.666524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:16.668758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:16.668758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:16.668758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:16.668758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:16.668758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:16.668758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:16.668758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:16.668758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:16.668758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:16.668758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:16.668758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:16.668758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:16.668758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:16.668758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:16.668758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:16.668758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:16.670436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:16.670436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:16.670436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:16.670436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:16.670436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:16.670436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:16.670436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:16.670436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:16.670436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:16.670436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:16.670436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:18:16.670436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:16.676889 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 62/124] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:18:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:16.677294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8a5b0-94db-4477-9d96-47650eacc676 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:16.714908 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d0fc9-9035-4f85-9f67-bf79860658a5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:16.767566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d5de3-ab41-4fa4-a0df-714c16df37c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:16.768302 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d5de3-ab41-4fa4-a0df-714c16df37c9 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:16.768596 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d5de3-ab41-4fa4-a0df-714c16df37c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:16.768884 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d5de3-ab41-4fa4-a0df-714c16df37c9 None admin] PATH_INFO: `/v3/users/5b20a8c5e8314c8794af566d108815a8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:16.770560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d5de3-ab41-4fa4-a0df-714c16df37c9 None admin] RBAC: Authorizing `identity:get_user(user_id=5b20a8c5e8314c8794af566d108815a8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:16.772468 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d5de3-ab41-4fa4-a0df-714c16df37c9 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:16.779981 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 63/125] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:16 2022] GET /identity/v3/users/5b20a8c5e8314c8794af566d108815a8 => generated 276 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:16.779981 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d5de3-ab41-4fa4-a0df-714c16df37c9 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:16.852400 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8a5b0-94db-4477-9d96-47650eacc676 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:16.892989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29bf8e-0feb-4238-869c-35a3490f5ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:16.893598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29bf8e-0feb-4238-869c-35a3490f5ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:16.893893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29bf8e-0feb-4238-869c-35a3490f5ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:16.894321 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29bf8e-0feb-4238-869c-35a3490f5ce7 None admin] PATH_INFO: `/v3/users/5b20a8c5e8314c8794af566d108815a8` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:16.895590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29bf8e-0feb-4238-869c-35a3490f5ce7 None admin] RBAC: Authorizing `identity:get_user(user_id=5b20a8c5e8314c8794af566d108815a8)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:16.898752 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29bf8e-0feb-4238-869c-35a3490f5ce7 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:16.907241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 63/126] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:16 2022] GET /identity/v3/users/5b20a8c5e8314c8794af566d108815a8 => generated 276 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:16.907241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29bf8e-0feb-4238-869c-35a3490f5ce7 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:16.916762 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:16.948655 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:16.954708 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d5de3-ab41-4fa4-a0df-714c16df37c9 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:17.008936 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983d4a3-a6f6-4001-b764-5e9204ae977c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:17.010213 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983d4a3-a6f6-4001-b764-5e9204ae977c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:17.010793 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983d4a3-a6f6-4001-b764-5e9204ae977c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:17.011317 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983d4a3-a6f6-4001-b764-5e9204ae977c None admin] PATH_INFO: `/v3/users/5b20a8c5e8314c8794af566d108815a8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:17.013107 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983d4a3-a6f6-4001-b764-5e9204ae977c None admin] RBAC: Authorizing `identity:get_user(user_id=5b20a8c5e8314c8794af566d108815a8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:17.015070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983d4a3-a6f6-4001-b764-5e9204ae977c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:17.017757 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 64/127] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:16 2022] GET /identity/v3/users/5b20a8c5e8314c8794af566d108815a8 => generated 276 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:17.018992 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983d4a3-a6f6-4001-b764-5e9204ae977c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:17.056320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29bf8e-0feb-4238-869c-35a3490f5ce7 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:17.100286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00ee6e-6810-4184-bb88-93efeb9f6020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:17.101094 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00ee6e-6810-4184-bb88-93efeb9f6020 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:17.101391 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00ee6e-6810-4184-bb88-93efeb9f6020 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:17.101679 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00ee6e-6810-4184-bb88-93efeb9f6020 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:17.102519 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a00ee6e-6810-4184-bb88-93efeb9f6020 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:17.102852 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a00ee6e-6810-4184-bb88-93efeb9f6020 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:17.104662 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a00ee6e-6810-4184-bb88-93efeb9f6020 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:17.115777 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 64/128] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:16 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:17.116652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00ee6e-6810-4184-bb88-93efeb9f6020 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:17.171716 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983d4a3-a6f6-4001-b764-5e9204ae977c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:17.214147 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de45438-4b58-4bd7-bc25-8d8ce8c5e227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:17.215608 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de45438-4b58-4bd7-bc25-8d8ce8c5e227 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:17.216457 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de45438-4b58-4bd7-bc25-8d8ce8c5e227 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:17.217202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de45438-4b58-4bd7-bc25-8d8ce8c5e227 None admin] PATH_INFO: `/v3/projects/6d7a7a0b84384f14849056212228badb` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:17.219507 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de45438-4b58-4bd7-bc25-8d8ce8c5e227 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7a7a0b84384f14849056212228badb)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:17.222115 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de45438-4b58-4bd7-bc25-8d8ce8c5e227 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:17.223568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:17.232440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 65/129] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:17 2022] GET /identity/v3/projects/6d7a7a0b84384f14849056212228badb => generated 299 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:17.233840 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de45438-4b58-4bd7-bc25-8d8ce8c5e227 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:17.262853 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00ee6e-6810-4184-bb88-93efeb9f6020 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:17.283794 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:17.303936 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf10a2b5-f2a1-453e-9aa2-7bf846d0c097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:17.304684 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10a2b5-f2a1-453e-9aa2-7bf846d0c097 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:17.305003 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10a2b5-f2a1-453e-9aa2-7bf846d0c097 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:17.305458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10a2b5-f2a1-453e-9aa2-7bf846d0c097 None admin] PATH_INFO: `/v3/projects/febd81c48dc747079c83424c156e5e27` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:17.306503 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf10a2b5-f2a1-453e-9aa2-7bf846d0c097 None admin] RBAC: Authorizing `identity:get_project(project_id=febd81c48dc747079c83424c156e5e27)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:17.308067 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf10a2b5-f2a1-453e-9aa2-7bf846d0c097 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:17.314933 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 65/130] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:17 2022] GET /identity/v3/projects/febd81c48dc747079c83424c156e5e27 => generated 313 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:17.314933 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf10a2b5-f2a1-453e-9aa2-7bf846d0c097 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:17.371118 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de45438-4b58-4bd7-bc25-8d8ce8c5e227 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:17.412229 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1a219-8b27-4241-860b-a664bbec43db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:17.413460 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1a219-8b27-4241-860b-a664bbec43db None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:17.413951 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1a219-8b27-4241-860b-a664bbec43db None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:17.414460 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1a219-8b27-4241-860b-a664bbec43db None admin] PATH_INFO: `/v3/projects/6d7a7a0b84384f14849056212228badb` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:17.416127 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd1a219-8b27-4241-860b-a664bbec43db None admin] RBAC: Authorizing `identity:get_project(project_id=6d7a7a0b84384f14849056212228badb)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:17.420185 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd1a219-8b27-4241-860b-a664bbec43db None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:17.424636 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 66/131] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:17 2022] GET /identity/v3/projects/6d7a7a0b84384f14849056212228badb => generated 299 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:17.425947 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1a219-8b27-4241-860b-a664bbec43db None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:17.430567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf10a2b5-f2a1-453e-9aa2-7bf846d0c097 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:17.470390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13fd371-1619-4138-b707-54c01aa19962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:17.471043 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fd371-1619-4138-b707-54c01aa19962 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:17.471323 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fd371-1619-4138-b707-54c01aa19962 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:17.471609 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fd371-1619-4138-b707-54c01aa19962 None admin] PATH_INFO: `/v3/users/4d1eaa7a05e745a5817149d8a56dab58` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:17.472983 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13fd371-1619-4138-b707-54c01aa19962 None admin] RBAC: Authorizing `identity:get_user(user_id=4d1eaa7a05e745a5817149d8a56dab58)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:17.474538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13fd371-1619-4138-b707-54c01aa19962 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:17.476983 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 66/132] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:17 2022] GET /identity/v3/users/4d1eaa7a05e745a5817149d8a56dab58 => generated 248 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:17.477175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13fd371-1619-4138-b707-54c01aa19962 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:17.486598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:17.602064 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1a219-8b27-4241-860b-a664bbec43db None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:17.605984 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:17.618660 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13fd371-1619-4138-b707-54c01aa19962 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:17.658383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014cab9d-98e7-4678-a38d-e208f95bd4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:17.659930 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cab9d-98e7-4678-a38d-e208f95bd4cc None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:17.660557 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cab9d-98e7-4678-a38d-e208f95bd4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:17.661310 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cab9d-98e7-4678-a38d-e208f95bd4cc None admin] PATH_INFO: `/v3/projects/6d7a7a0b84384f14849056212228badb/users/5b20a8c5e8314c8794af566d108815a8/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:17.661643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de3cb8-a67b-4c45-b2c5-850383249018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:17.662449 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de3cb8-a67b-4c45-b2c5-850383249018 None admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:17.662747 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de3cb8-a67b-4c45-b2c5-850383249018 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:17.663032 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de3cb8-a67b-4c45-b2c5-850383249018 None admin] PATH_INFO: `/v3/projects/febd81c48dc747079c83424c156e5e27/users/5b20a8c5e8314c8794af566d108815a8/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:17.665125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014cab9d-98e7-4678-a38d-e208f95bd4cc None admin] RBAC: Authorizing `identity:create_grant(project_id=6d7a7a0b84384f14849056212228badb, user_id=5b20a8c5e8314c8794af566d108815a8, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:17.665280 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de3cb8-a67b-4c45-b2c5-850383249018 None admin] RBAC: Authorizing `identity:create_grant(project_id=febd81c48dc747079c83424c156e5e27, user_id=5b20a8c5e8314c8794af566d108815a8, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:17.666820 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de3cb8-a67b-4c45-b2c5-850383249018 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:17.667526 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014cab9d-98e7-4678-a38d-e208f95bd4cc None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:17.687760 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 67/133] 10.222.0.121 () {58 vars in 1444 bytes} [Thu Aug 4 10:18:17 2022] PUT /identity/v3/projects/6d7a7a0b84384f14849056212228badb/users/5b20a8c5e8314c8794af566d108815a8/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:18:17.694282 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 67/134] 10.222.0.121 () {58 vars in 1444 bytes} [Thu Aug 4 10:18:17 2022] PUT /identity/v3/projects/febd81c48dc747079c83424c156e5e27/users/5b20a8c5e8314c8794af566d108815a8/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:18:17.694282 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de3cb8-a67b-4c45-b2c5-850383249018 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:17.694633 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014cab9d-98e7-4678-a38d-e208f95bd4cc None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:17.778475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:17.855815 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de3cb8-a67b-4c45-b2c5-850383249018 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:17.929597 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014cab9d-98e7-4678-a38d-e208f95bd4cc None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:17.950147 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8c89b-09b4-4926-b3d5-ca7034411b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:17.951289 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8c89b-09b4-4926-b3d5-ca7034411b78 None admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:17.951289 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8c89b-09b4-4926-b3d5-ca7034411b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:17.951589 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8c89b-09b4-4926-b3d5-ca7034411b78 None admin] PATH_INFO: `/v3/projects/6d7a7a0b84384f14849056212228badb/users/5b20a8c5e8314c8794af566d108815a8/roles/89b847fca3574b9083287279aee0f773` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:17.954026 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e8c89b-09b4-4926-b3d5-ca7034411b78 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d7a7a0b84384f14849056212228badb, user_id=5b20a8c5e8314c8794af566d108815a8, role_id=89b847fca3574b9083287279aee0f773)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:17.955850 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e8c89b-09b4-4926-b3d5-ca7034411b78 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:17.985209 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 68/135] 10.222.0.121 () {58 vars in 1444 bytes} [Thu Aug 4 10:18:17 2022] PUT /identity/v3/projects/6d7a7a0b84384f14849056212228badb/users/5b20a8c5e8314c8794af566d108815a8/roles/89b847fca3574b9083287279aee0f773 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:18:17.987035 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff4cb6-556e-4d46-81ca-14ab2a88867a None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:17.987358 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbac53-fa59-4552-9e62-d17afe158481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:17.987852 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff4cb6-556e-4d46-81ca-14ab2a88867a None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:17.987852 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff4cb6-556e-4d46-81ca-14ab2a88867a None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:17.989297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbac53-fa59-4552-9e62-d17afe158481 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:17.989906 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbac53-fa59-4552-9e62-d17afe158481 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:17.990477 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbac53-fa59-4552-9e62-d17afe158481 None admin] PATH_INFO: `/v3/projects/6d7a7a0b84384f14849056212228badb` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:17.992438 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbac53-fa59-4552-9e62-d17afe158481 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7a7a0b84384f14849056212228badb)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:17.993481 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 69/136] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:17 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:17.994514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbac53-fa59-4552-9e62-d17afe158481 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:17.996837 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 68/137] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:17 2022] GET /identity/v3/projects/6d7a7a0b84384f14849056212228badb => generated 299 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:18.003509 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90913f1a-fe66-441e-a98d-27154fdb0d11 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:18.003735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90913f1a-fe66-441e-a98d-27154fdb0d11 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:18.004040 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90913f1a-fe66-441e-a98d-27154fdb0d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:18.011073 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbac53-fa59-4552-9e62-d17afe158481 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:18.031023 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-90913f1a-fe66-441e-a98d-27154fdb0d11 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:18.032554 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-90913f1a-fe66-441e-a98d-27154fdb0d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:18.064900 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:18.129093 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 70/138] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:18 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:18.159200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90913f1a-fe66-441e-a98d-27154fdb0d11 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:18.175299 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbac53-fa59-4552-9e62-d17afe158481 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:18.230339 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:18.259668 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643df603-a039-4288-a27d-779547dd5db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:18.261407 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643df603-a039-4288-a27d-779547dd5db8 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:18.261704 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643df603-a039-4288-a27d-779547dd5db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:18.261993 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643df603-a039-4288-a27d-779547dd5db8 None admin] PATH_INFO: `/v3/projects/6d7a7a0b84384f14849056212228badb/users/4d1eaa7a05e745a5817149d8a56dab58/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:18.275359 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643df603-a039-4288-a27d-779547dd5db8 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d7a7a0b84384f14849056212228badb, user_id=4d1eaa7a05e745a5817149d8a56dab58, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:18.277240 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643df603-a039-4288-a27d-779547dd5db8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:18.306639 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 69/139] 10.222.0.121 () {58 vars in 1444 bytes} [Thu Aug 4 10:18:18 2022] PUT /identity/v3/projects/6d7a7a0b84384f14849056212228badb/users/4d1eaa7a05e745a5817149d8a56dab58/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:18:18.349429 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90913f1a-fe66-441e-a98d-27154fdb0d11 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:18.411690 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3654595-e10f-4059-ac9c-b9b5a1014839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:18.412462 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3654595-e10f-4059-ac9c-b9b5a1014839 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:18.412755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3654595-e10f-4059-ac9c-b9b5a1014839 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:18.413044 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3654595-e10f-4059-ac9c-b9b5a1014839 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:18.414536 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3654595-e10f-4059-ac9c-b9b5a1014839 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:18.416417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3654595-e10f-4059-ac9c-b9b5a1014839 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:18.418597 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 71/140] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:18:18 2022] GET /identity/v3/domains/default => generated 200 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:18.427362 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643df603-a039-4288-a27d-779547dd5db8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:18.428186 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc935c6-c92e-4055-a309-63a3323aa7b0 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:18.428629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc935c6-c92e-4055-a309-63a3323aa7b0 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:18.429097 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc935c6-c92e-4055-a309-63a3323aa7b0 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:18.431194 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 72/141] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:18 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:18.440454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c3caf-10ee-4fc0-b1f2-723c10c2075c None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:18.440775 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c3caf-10ee-4fc0-b1f2-723c10c2075c None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:18.441169 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c3caf-10ee-4fc0-b1f2-723c10c2075c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:18.468948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-762c3caf-10ee-4fc0-b1f2-723c10c2075c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:18.470591 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-762c3caf-10ee-4fc0-b1f2-723c10c2075c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:18.567227 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 73/142] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:18 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 04 10:18:18.609652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c3caf-10ee-4fc0-b1f2-723c10c2075c None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:18.692323 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:18.697918 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:18.711223 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643df603-a039-4288-a27d-779547dd5db8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:18.762226 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c3caf-10ee-4fc0-b1f2-723c10c2075c None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:18.802957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426b14c-aa8a-4608-b25d-78a744bdc181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:18.804239 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426b14c-aa8a-4608-b25d-78a744bdc181 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:18.804769 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426b14c-aa8a-4608-b25d-78a744bdc181 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:18.805270 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426b14c-aa8a-4608-b25d-78a744bdc181 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:18.806488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426b14c-aa8a-4608-b25d-78a744bdc181 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:18.807577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbc993-3dcd-4836-a838-09cede49850a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:18.808346 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbc993-3dcd-4836-a838-09cede49850a None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:18.808680 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbc993-3dcd-4836-a838-09cede49850a None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:18.808963 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbc993-3dcd-4836-a838-09cede49850a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:18.809248 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426b14c-aa8a-4608-b25d-78a744bdc181 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:18.822225 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cbc993-3dcd-4836-a838-09cede49850a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:18.829767 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-27cbc993-3dcd-4836-a838-09cede49850a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:18:18.829767 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:18.829767 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:18.829767 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:18.829767 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:18.829767 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:18.829767 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:18.829767 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:18.829767 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:18.829767 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:18.829767 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:18.829767 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:18.829767 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:18.829767 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:18.829767 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:18.829767 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:18.835530 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:18.835530 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:18.835530 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:18.835530 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:18.835530 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:18.835530 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:18.835530 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:18.835530 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:18.835530 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:18.835530 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:18.835530 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:18.835530 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:18.835530 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:18.835530 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:18.835530 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:18.835530 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:18.836582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:18.836582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:18.836582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:18.836582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:18.836582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:18.836582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:18.836582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:18.836582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:18.836582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:18.836582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:18.836582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:18:18.836582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:18.836582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 74/143] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:18:18.839254 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbc993-3dcd-4836-a838-09cede49850a None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:18.872854 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 70/144] 10.222.0.121 () {60 vars in 1137 bytes} [Thu Aug 4 10:18:18 2022] POST /identity/v3/users => generated 284 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:18:18.988369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbc993-3dcd-4836-a838-09cede49850a None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:19.005850 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:19.036195 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6497d6a5-31d9-4e38-8371-ff5e5271dba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:19.036938 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6497d6a5-31d9-4e38-8371-ff5e5271dba6 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:19.037228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6497d6a5-31d9-4e38-8371-ff5e5271dba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:19.037512 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6497d6a5-31d9-4e38-8371-ff5e5271dba6 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:19.038365 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6497d6a5-31d9-4e38-8371-ff5e5271dba6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:19.038709 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6497d6a5-31d9-4e38-8371-ff5e5271dba6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:19.040317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6497d6a5-31d9-4e38-8371-ff5e5271dba6 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:19.052218 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 75/145] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:18 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:19.059082 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426b14c-aa8a-4608-b25d-78a744bdc181 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:19.257497 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426b14c-aa8a-4608-b25d-78a744bdc181 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:19.309860 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34899e33-9d45-4281-95e0-2826c9b7395e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:19.310586 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34899e33-9d45-4281-95e0-2826c9b7395e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:19.310898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34899e33-9d45-4281-95e0-2826c9b7395e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:19.311186 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34899e33-9d45-4281-95e0-2826c9b7395e None admin] PATH_INFO: `/v3/users/d7693ebbe28d47628c3a1ec1cdff6144` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:19.312686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34899e33-9d45-4281-95e0-2826c9b7395e None admin] RBAC: Authorizing `identity:get_user(user_id=d7693ebbe28d47628c3a1ec1cdff6144)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:19.314721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34899e33-9d45-4281-95e0-2826c9b7395e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:19.317135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 71/146] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:19 2022] GET /identity/v3/users/d7693ebbe28d47628c3a1ec1cdff6144 => generated 290 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:19.323992 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6497d6a5-31d9-4e38-8371-ff5e5271dba6 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:19.578948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6497d6a5-31d9-4e38-8371-ff5e5271dba6 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:19.661634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de227803-e861-4f40-8b71-74a46483eb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:19.662714 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de227803-e861-4f40-8b71-74a46483eb5f None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:19.663310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de227803-e861-4f40-8b71-74a46483eb5f None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:19.663768 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de227803-e861-4f40-8b71-74a46483eb5f None admin] PATH_INFO: `/v3/projects/6d7a7a0b84384f14849056212228badb` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:19.666131 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de227803-e861-4f40-8b71-74a46483eb5f None admin] RBAC: Authorizing `identity:get_project(project_id=6d7a7a0b84384f14849056212228badb)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:19.668981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de227803-e861-4f40-8b71-74a46483eb5f None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:19.672292 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 76/147] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:19 2022] GET /identity/v3/projects/6d7a7a0b84384f14849056212228badb => generated 299 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:19.679523 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34899e33-9d45-4281-95e0-2826c9b7395e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:19.717156 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:19.885540 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34899e33-9d45-4281-95e0-2826c9b7395e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:19.943938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3381930-7c63-472e-8a71-cf10dddc1a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:19.945145 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3381930-7c63-472e-8a71-cf10dddc1a03 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:19.945631 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3381930-7c63-472e-8a71-cf10dddc1a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:19.946120 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3381930-7c63-472e-8a71-cf10dddc1a03 None admin] PATH_INFO: `/v3/projects/6d7a7a0b84384f14849056212228badb/users/d7693ebbe28d47628c3a1ec1cdff6144/roles/c77f1c570e8640a3b2c836a401b91889` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:19.949599 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3381930-7c63-472e-8a71-cf10dddc1a03 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d7a7a0b84384f14849056212228badb, user_id=d7693ebbe28d47628c3a1ec1cdff6144, role_id=c77f1c570e8640a3b2c836a401b91889)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:19.951417 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3381930-7c63-472e-8a71-cf10dddc1a03 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:19.975650 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 72/148] 10.222.0.121 () {58 vars in 1444 bytes} [Thu Aug 4 10:18:19 2022] PUT /identity/v3/projects/6d7a7a0b84384f14849056212228badb/users/d7693ebbe28d47628c3a1ec1cdff6144/roles/c77f1c570e8640a3b2c836a401b91889 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:18:21.731852 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d28eb-5a80-4d21-9292-f4f0bd4a6f1a None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:21.732727 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d28eb-5a80-4d21-9292-f4f0bd4a6f1a None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:21.732727 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d28eb-5a80-4d21-9292-f4f0bd4a6f1a None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:21.734439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 77/149] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:21 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:21.744374 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa522db-4ef9-4308-a1ca-1cfb2cc4219b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:21.745387 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa522db-4ef9-4308-a1ca-1cfb2cc4219b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:21.745387 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa522db-4ef9-4308-a1ca-1cfb2cc4219b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:21.774431 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4aa522db-4ef9-4308-a1ca-1cfb2cc4219b None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:21.775917 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4aa522db-4ef9-4308-a1ca-1cfb2cc4219b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:21.894294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 73/150] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:21 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 04 10:18:21.944504 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d28eb-5a80-4d21-9292-f4f0bd4a6f1a None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:22.012378 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:22.111540 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d28eb-5a80-4d21-9292-f4f0bd4a6f1a None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:22.166600 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd3b08-71dd-4891-a3ce-145d45d1699e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:22.167223 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd3b08-71dd-4891-a3ce-145d45d1699e None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:22.167552 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd3b08-71dd-4891-a3ce-145d45d1699e None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:22.167882 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd3b08-71dd-4891-a3ce-145d45d1699e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:22.186250 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcd3b08-71dd-4891-a3ce-145d45d1699e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:22.197051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-5fcd3b08-71dd-4891-a3ce-145d45d1699e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:22.197051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:22.197051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:22.197051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:22.197051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:22.197051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:22.197051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:22.197051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:22.197051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:22.197051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:22.197051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:22.197051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:22.197051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:22.197051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:22.197051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:22.197051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:22.198972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:22.198972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:22.198972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:22.198972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:22.198972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:22.198972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:22.198972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:22.198972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:22.198972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:22.198972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:22.198972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:22.198972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:22.198972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:22.198972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:22.198972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:22.198972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:22.201892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:22.201892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:22.201892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:22.201892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:22.201892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:22.201892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:22.201892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:22.201892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:22.201892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:22.201892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:22.201892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:22.201892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:22.201892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 78/151] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:22.209283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa522db-4ef9-4308-a1ca-1cfb2cc4219b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:22.225330 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:22.377657 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6503ed6-ad4c-4610-b8bc-c13d3214b937 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:22.380523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6503ed6-ad4c-4610-b8bc-c13d3214b937 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:22.380917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6503ed6-ad4c-4610-b8bc-c13d3214b937 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:22.383234 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 79/152] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:22 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:22.392997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f256e68-73b9-48b6-83f6-6222bde22de9 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:22.393284 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f256e68-73b9-48b6-83f6-6222bde22de9 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:22.393689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f256e68-73b9-48b6-83f6-6222bde22de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:22.418985 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa522db-4ef9-4308-a1ca-1cfb2cc4219b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:22.429620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-6f256e68-73b9-48b6-83f6-6222bde22de9 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:22.430896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6f256e68-73b9-48b6-83f6-6222bde22de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:22.515728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d8f3e-d772-4444-a6b1-ceb0ae6b8c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:22.516683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d8f3e-d772-4444-a6b1-ceb0ae6b8c30 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:22.517124 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d8f3e-d772-4444-a6b1-ceb0ae6b8c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:22.517552 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d8f3e-d772-4444-a6b1-ceb0ae6b8c30 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:22.518619 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59d8f3e-d772-4444-a6b1-ceb0ae6b8c30 None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:22.519014 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59d8f3e-d772-4444-a6b1-ceb0ae6b8c30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:22.520820 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59d8f3e-d772-4444-a6b1-ceb0ae6b8c30 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:22.546156 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 74/153] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:22 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:22.573610 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d8f3e-d772-4444-a6b1-ceb0ae6b8c30 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:22.615294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 80/154] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:22 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:18:22.643589 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeaf5aa-9090-4342-bedd-7b054cddc63e None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:22.648529 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeaf5aa-9090-4342-bedd-7b054cddc63e None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:22.649071 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeaf5aa-9090-4342-bedd-7b054cddc63e None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:22.651251 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 81/155] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:22 2022] GET /identity => generated 271 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:22.656454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeaf5aa-9090-4342-bedd-7b054cddc63e None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:22.752497 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:22.758958 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:22.843698 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d8f3e-d772-4444-a6b1-ceb0ae6b8c30 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:22.949150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7bea74-a9e7-41ff-987e-face4ad6c758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:22.950889 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bea74-a9e7-41ff-987e-face4ad6c758 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:22.951714 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bea74-a9e7-41ff-987e-face4ad6c758 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:22.952518 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bea74-a9e7-41ff-987e-face4ad6c758 None admin] PATH_INFO: `/v3/users/5b20a8c5e8314c8794af566d108815a8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:22.956583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7bea74-a9e7-41ff-987e-face4ad6c758 None admin] RBAC: Authorizing `identity:get_user(user_id=5b20a8c5e8314c8794af566d108815a8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:22.959637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7bea74-a9e7-41ff-987e-face4ad6c758 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:22.963640 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 75/156] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:22 2022] GET /identity/v3/users/5b20a8c5e8314c8794af566d108815a8 => generated 276 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:22.966292 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb8c4d-b940-47b0-b8e5-2aa27f2c7ca8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:22.967162 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb8c4d-b940-47b0-b8e5-2aa27f2c7ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:22.972711 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb8c4d-b940-47b0-b8e5-2aa27f2c7ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:22.993374 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeaf5aa-9090-4342-bedd-7b054cddc63e None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:23.025934 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-62fb8c4d-b940-47b0-b8e5-2aa27f2c7ca8 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:23.028692 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-62fb8c4d-b940-47b0-b8e5-2aa27f2c7ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:23.070790 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67542930-87c9-4912-8b81-dd45f531ed89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:23.072307 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67542930-87c9-4912-8b81-dd45f531ed89 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:23.073120 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67542930-87c9-4912-8b81-dd45f531ed89 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:23.073784 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67542930-87c9-4912-8b81-dd45f531ed89 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:23.097834 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67542930-87c9-4912-8b81-dd45f531ed89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:23.111932 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-67542930-87c9-4912-8b81-dd45f531ed89 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:18:23.111932 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:23.111932 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:23.111932 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:23.111932 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:23.111932 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:23.111932 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:23.111932 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:23.111932 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:23.111932 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:23.111932 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:23.111932 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:23.111932 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:23.111932 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:23.111932 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:23.111932 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:23.115408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:23.115408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:23.115408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:23.115408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:23.115408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:23.115408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:23.115408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:23.115408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:23.115408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:23.115408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:23.115408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:23.115408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:23.115408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:23.115408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:23.115408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:23.115408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:23.117869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:23.117869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:23.117869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:23.117869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:23.117869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:23.117869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:23.117869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:23.117869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:23.117869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:23.117869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:23.117869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:18:23.117869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:23.122869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 82/157] 10.222.0.121 () {56 vars in 1119 bytes} [Thu Aug 4 10:18:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 471 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:23.125288 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67542930-87c9-4912-8b81-dd45f531ed89 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:23.211098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 76/158] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:22 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:23.212084 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb8c4d-b940-47b0-b8e5-2aa27f2c7ca8 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:23.336612 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:23.418324 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb8c4d-b940-47b0-b8e5-2aa27f2c7ca8 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:23.492446 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:23.497812 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0136fc9-3547-4b90-b8f7-afd629c4df3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:23.499176 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0136fc9-3547-4b90-b8f7-afd629c4df3c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:23.499477 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0136fc9-3547-4b90-b8f7-afd629c4df3c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:23.499772 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0136fc9-3547-4b90-b8f7-afd629c4df3c None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:23.501090 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0136fc9-3547-4b90-b8f7-afd629c4df3c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:23.501520 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0136fc9-3547-4b90-b8f7-afd629c4df3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:23.503347 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0136fc9-3547-4b90-b8f7-afd629c4df3c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:23.524220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 77/159] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:18:23 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:23.525801 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0136fc9-3547-4b90-b8f7-afd629c4df3c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:23.540909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67542930-87c9-4912-8b81-dd45f531ed89 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:23.730573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7f790-5e5f-4ce4-9dc0-4e5a5352aec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:23.731686 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7f790-5e5f-4ce4-9dc0-4e5a5352aec1 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:23.732229 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7f790-5e5f-4ce4-9dc0-4e5a5352aec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:23.732733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7f790-5e5f-4ce4-9dc0-4e5a5352aec1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:23.805180 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0136fc9-3547-4b90-b8f7-afd629c4df3c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:23.884877 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:23.906752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282cdae6-1525-49f5-afbc-4fd077c2535d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:23.907907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282cdae6-1525-49f5-afbc-4fd077c2535d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:23.908413 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282cdae6-1525-49f5-afbc-4fd077c2535d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:23.908865 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282cdae6-1525-49f5-afbc-4fd077c2535d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:23.952769 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282cdae6-1525-49f5-afbc-4fd077c2535d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:23.973360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-282cdae6-1525-49f5-afbc-4fd077c2535d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:23.973360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:23.973360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:23.973360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:23.973360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:23.973360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:23.973360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:23.973360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:23.973360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:23.973360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:23.973360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:23.973360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:23.973360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:23.973360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:23.973360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:23.973360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:23.974646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:23.974646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:23.974646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:23.974646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:23.974646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:23.974646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:23.974646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:23.974646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:23.974646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:23.974646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:23.974646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:23.974646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:23.974646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:23.974646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:23.974646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:23.974646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:23.977979 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:23.977979 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:23.977979 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:23.977979 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:23.977979 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:23.977979 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:23.977979 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:23.977979 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:23.977979 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:23.977979 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:23.977979 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:23.977979 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:18:23.977979 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 78/160] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 451 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:23.979724 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbb659-f430-41cc-87e6-064ed02d8f7c None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:23.988948 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbb659-f430-41cc-87e6-064ed02d8f7c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:23.989570 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbb659-f430-41cc-87e6-064ed02d8f7c None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:23.993424 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 79/161] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:23 2022] GET /identity => generated 271 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:23.995149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbb659-f430-41cc-87e6-064ed02d8f7c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:23.998451 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7f790-5e5f-4ce4-9dc0-4e5a5352aec1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:24.001289 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7f790-5e5f-4ce4-9dc0-4e5a5352aec1 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:24.299380 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:24.334171 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 83/162] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:18:23 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 1210 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:18:24.335549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7f790-5e5f-4ce4-9dc0-4e5a5352aec1 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:24.357829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbb659-f430-41cc-87e6-064ed02d8f7c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:24.508371 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cc9bc-aa00-49d8-b5a1-e0ce109e0af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:24.510539 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cc9bc-aa00-49d8-b5a1-e0ce109e0af4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:24.511088 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cc9bc-aa00-49d8-b5a1-e0ce109e0af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:24.511668 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cc9bc-aa00-49d8-b5a1-e0ce109e0af4 None admin] PATH_INFO: `/v3/users/5b20a8c5e8314c8794af566d108815a8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:24.515383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9cc9bc-aa00-49d8-b5a1-e0ce109e0af4 None admin] RBAC: Authorizing `identity:get_user(user_id=5b20a8c5e8314c8794af566d108815a8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:24.519710 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9cc9bc-aa00-49d8-b5a1-e0ce109e0af4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:24.525643 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 80/163] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:23 2022] GET /identity/v3/users/5b20a8c5e8314c8794af566d108815a8 => generated 276 bytes in 531 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:24.528487 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c728ba-06d8-4083-b858-e647e6b7daba None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:24.529049 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c728ba-06d8-4083-b858-e647e6b7daba None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:24.529560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c728ba-06d8-4083-b858-e647e6b7daba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:24.580581 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c7c728ba-06d8-4083-b858-e647e6b7daba None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:24.583855 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c7c728ba-06d8-4083-b858-e647e6b7daba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:24.670709 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7f790-5e5f-4ce4-9dc0-4e5a5352aec1 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:24.725295 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:24.759923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca1ccb-8010-4ef7-996e-1be442ecbf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:24.761806 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca1ccb-8010-4ef7-996e-1be442ecbf73 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:24.761806 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca1ccb-8010-4ef7-996e-1be442ecbf73 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:24.761806 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca1ccb-8010-4ef7-996e-1be442ecbf73 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:24.762644 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca1ccb-8010-4ef7-996e-1be442ecbf73 None admin] RBAC: Adding query filter params (name=member) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:24.762806 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca1ccb-8010-4ef7-996e-1be442ecbf73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:24.764753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca1ccb-8010-4ef7-996e-1be442ecbf73 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:24.768452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 81/164] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:24 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:24.769694 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13c171-8363-49c4-bee1-56aac4d7a35b None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:24.770244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13c171-8363-49c4-bee1-56aac4d7a35b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:24.770386 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13c171-8363-49c4-bee1-56aac4d7a35b None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:24.771756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 82/165] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:24 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:24.777437 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13c171-8363-49c4-bee1-56aac4d7a35b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:24.782960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 84/166] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:24 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 449 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:24.783976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca1ccb-8010-4ef7-996e-1be442ecbf73 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:24.901284 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:24.965182 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca1ccb-8010-4ef7-996e-1be442ecbf73 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:24.979224 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13c171-8363-49c4-bee1-56aac4d7a35b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:25.008112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d91e11-15df-420e-826f-e39c8cacd591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:25.008902 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d91e11-15df-420e-826f-e39c8cacd591 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:25.009137 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d91e11-15df-420e-826f-e39c8cacd591 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:25.010019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d91e11-15df-420e-826f-e39c8cacd591 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:25.021219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac551b-4110-4b4e-949f-b379a43d1e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:25.021944 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac551b-4110-4b4e-949f-b379a43d1e1a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:25.022284 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac551b-4110-4b4e-949f-b379a43d1e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:25.022733 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac551b-4110-4b4e-949f-b379a43d1e1a None admin] PATH_INFO: `/v3/projects/febd81c48dc747079c83424c156e5e27` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:25.024144 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac551b-4110-4b4e-949f-b379a43d1e1a None admin] RBAC: Authorizing `identity:get_project(project_id=febd81c48dc747079c83424c156e5e27)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:25.025789 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac551b-4110-4b4e-949f-b379a43d1e1a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:25.033646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 83/167] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:24 2022] GET /identity/v3/projects/febd81c48dc747079c83424c156e5e27 => generated 313 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:25.033646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2585de3-8a32-4f75-bf93-d4a1e4fccd54 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:25.033646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2585de3-8a32-4f75-bf93-d4a1e4fccd54 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:25.033646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2585de3-8a32-4f75-bf93-d4a1e4fccd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:25.052313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d2585de3-8a32-4f75-bf93-d4a1e4fccd54 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:25.053942 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d2585de3-8a32-4f75-bf93-d4a1e4fccd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:25.114397 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d91e11-15df-420e-826f-e39c8cacd591 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:25.117386 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d91e11-15df-420e-826f-e39c8cacd591 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:25.159590 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:25.189812 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:25.198173 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 84/168] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:25 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:25.205110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2585de3-8a32-4f75-bf93-d4a1e4fccd54 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:25.272404 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 85/169] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:18:24 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:18:25.272404 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d91e11-15df-420e-826f-e39c8cacd591 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:25.379566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2585de3-8a32-4f75-bf93-d4a1e4fccd54 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:25.404147 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d91e11-15df-420e-826f-e39c8cacd591 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:25.419825 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757faca1-b3ea-4029-b5ab-7d66752b168a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:25.420829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757faca1-b3ea-4029-b5ab-7d66752b168a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:25.421292 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757faca1-b3ea-4029-b5ab-7d66752b168a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:25.421752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757faca1-b3ea-4029-b5ab-7d66752b168a None admin] PATH_INFO: `/v3/users/5b20a8c5e8314c8794af566d108815a8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:25.423734 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757faca1-b3ea-4029-b5ab-7d66752b168a None admin] RBAC: Authorizing `identity:get_user(user_id=5b20a8c5e8314c8794af566d108815a8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:25.426393 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757faca1-b3ea-4029-b5ab-7d66752b168a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:25.429217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 85/170] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:25 2022] GET /identity/v3/users/5b20a8c5e8314c8794af566d108815a8 => generated 276 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:25.431295 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e1252-df3d-4f83-95ce-72ff30de2805 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:25.432424 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e1252-df3d-4f83-95ce-72ff30de2805 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:25.432424 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e1252-df3d-4f83-95ce-72ff30de2805 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:25.434208 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 86/171] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:25 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:25.436197 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6e405-217d-4bc4-90e4-7cc62854e6fa None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:25.436430 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6e405-217d-4bc4-90e4-7cc62854e6fa None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:25.436804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6e405-217d-4bc4-90e4-7cc62854e6fa None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:25.439026 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 87/172] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:25 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:25.440554 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6e405-217d-4bc4-90e4-7cc62854e6fa None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:25.469578 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f22c8-af3f-44b5-a043-b79b844245a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:25.470583 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f22c8-af3f-44b5-a043-b79b844245a4 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:25.470975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f22c8-af3f-44b5-a043-b79b844245a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:25.471399 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f22c8-af3f-44b5-a043-b79b844245a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:25.485689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f22c8-af3f-44b5-a043-b79b844245a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:25.491390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-922f22c8-af3f-44b5-a043-b79b844245a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:18:25.491390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:25.491390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:25.491390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:25.491390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:25.491390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:25.491390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:25.491390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:25.491390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:25.491390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:25.491390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:25.491390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:25.491390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:25.491390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:25.491390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:25.491390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:25.492692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:25.492692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:25.492692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:25.492692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:25.492692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:25.492692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:25.492692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:25.492692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:25.492692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:25.492692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:25.492692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:25.492692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:25.492692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:25.492692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:25.492692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:25.492692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:25.494189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:25.494189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:25.494189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:25.494189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:25.494189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:25.494189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:25.494189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:25.494189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:25.494189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:25.494189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:25.494189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:18:25.494189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:25.494189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 86/173] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:18:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:25.494189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f22c8-af3f-44b5-a043-b79b844245a4 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:25.513417 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:25.520590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:25.583172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6e405-217d-4bc4-90e4-7cc62854e6fa None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:25.626289 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c52ba-26c8-4ae2-b2a0-1b41290e34ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:25.626932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c52ba-26c8-4ae2-b2a0-1b41290e34ea None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:25.627213 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c52ba-26c8-4ae2-b2a0-1b41290e34ea None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:25.627487 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c52ba-26c8-4ae2-b2a0-1b41290e34ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:25.628341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c52ba-26c8-4ae2-b2a0-1b41290e34ea None admin] RBAC: Adding query filter params (role.id=1b988376d12749bd8d653baa89493c5c, scope.project.id=febd81c48dc747079c83424c156e5e27, user.id=5b20a8c5e8314c8794af566d108815a8) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:25.628655 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c52ba-26c8-4ae2-b2a0-1b41290e34ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:25.630197 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c52ba-26c8-4ae2-b2a0-1b41290e34ea None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:25.637264 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 88/174] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:18:25 2022] GET /identity/v3/role_assignments?user.id=5b20a8c5e8314c8794af566d108815a8&scope.project.id=febd81c48dc747079c83424c156e5e27&role.id=1b988376d12749bd8d653baa89493c5c => generated 610 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:25.638234 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c52ba-26c8-4ae2-b2a0-1b41290e34ea None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:25.647725 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f22c8-af3f-44b5-a043-b79b844245a4 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:25.712402 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31941de8-be26-4304-b3c2-c0a5123a53e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:25.713130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31941de8-be26-4304-b3c2-c0a5123a53e1 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:25.713667 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31941de8-be26-4304-b3c2-c0a5123a53e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:25.714259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31941de8-be26-4304-b3c2-c0a5123a53e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:25.730278 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31941de8-be26-4304-b3c2-c0a5123a53e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:25.737064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-31941de8-be26-4304-b3c2-c0a5123a53e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:18:25.737064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:25.737064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:25.737064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:25.737064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:25.737064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:25.737064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:25.737064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:25.737064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:25.737064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:25.737064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:25.737064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:25.737064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:25.737064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:25.737064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:25.737064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:25.739349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:25.739349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:25.739349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:25.739349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:25.739349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:25.739349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:25.739349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:25.739349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:25.739349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:25.739349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:25.739349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:25.739349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:25.739349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:25.739349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:25.739349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:25.739349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:25.741371 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:25.741371 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:25.741371 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:25.741371 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:25.741371 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:25.741371 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:25.741371 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:25.741371 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:25.741371 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:25.741371 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:25.741371 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:18:25.741371 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:25.741371 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 87/175] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:18:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:25.741371 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00f15f-997a-4056-a133-fba4a1ecc97a None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:25.742894 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00f15f-997a-4056-a133-fba4a1ecc97a None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:25.742894 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00f15f-997a-4056-a133-fba4a1ecc97a None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:25.750244 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 88/176] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:25 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:25.750244 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00f15f-997a-4056-a133-fba4a1ecc97a None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:25.837656 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c52ba-26c8-4ae2-b2a0-1b41290e34ea None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:25.871337 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:25.877127 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:25.895674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56747b3-18bc-4047-bcd6-e13719dec669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:25.896427 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56747b3-18bc-4047-bcd6-e13719dec669 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:25.896720 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56747b3-18bc-4047-bcd6-e13719dec669 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:25.897025 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56747b3-18bc-4047-bcd6-e13719dec669 None admin] PATH_INFO: `/v3/projects/6d7a7a0b84384f14849056212228badb` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:25.898433 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56747b3-18bc-4047-bcd6-e13719dec669 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7a7a0b84384f14849056212228badb)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:25.899975 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56747b3-18bc-4047-bcd6-e13719dec669 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:25.901793 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 89/177] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:25 2022] GET /identity/v3/projects/6d7a7a0b84384f14849056212228badb => generated 299 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:25.903118 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b535f3-59c3-4129-80ef-ba2eef02b263 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:25.903501 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b535f3-59c3-4129-80ef-ba2eef02b263 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:25.903828 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b535f3-59c3-4129-80ef-ba2eef02b263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:25.924030 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c2b535f3-59c3-4129-80ef-ba2eef02b263 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:25.926314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c2b535f3-59c3-4129-80ef-ba2eef02b263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:25.929878 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00f15f-997a-4056-a133-fba4a1ecc97a None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:25.982236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b96705-67a1-40d9-891c-f04b2517ebcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:25.982867 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b96705-67a1-40d9-891c-f04b2517ebcd None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:25.983189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b96705-67a1-40d9-891c-f04b2517ebcd None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:25.983581 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b96705-67a1-40d9-891c-f04b2517ebcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:26.015872 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 90/178] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:25 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:26.018837 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6f92f-bd59-4fc7-95d1-b15d4f6760cd None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:26.019311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6f92f-bd59-4fc7-95d1-b15d4f6760cd None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:26.019881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6f92f-bd59-4fc7-95d1-b15d4f6760cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:26.047976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4aa6f92f-bd59-4fc7-95d1-b15d4f6760cd None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:26.049610 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4aa6f92f-bd59-4fc7-95d1-b15d4f6760cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:26.065619 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b96705-67a1-40d9-891c-f04b2517ebcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:26.067396 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b96705-67a1-40d9-891c-f04b2517ebcd None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:26.129842 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 91/179] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:26 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:26.130729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6f92f-bd59-4fc7-95d1-b15d4f6760cd None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:26.164337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:26.196357 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:26.214655 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 89/180] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:18:25 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:18:26.216088 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b96705-67a1-40d9-891c-f04b2517ebcd None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:26.270935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6f92f-bd59-4fc7-95d1-b15d4f6760cd None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:26.316638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0dbfab-219c-4dff-a359-64fd3f5f479d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:26.317344 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0dbfab-219c-4dff-a359-64fd3f5f479d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:26.317725 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0dbfab-219c-4dff-a359-64fd3f5f479d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:26.318008 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0dbfab-219c-4dff-a359-64fd3f5f479d None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:26.318960 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0dbfab-219c-4dff-a359-64fd3f5f479d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:26.319340 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0dbfab-219c-4dff-a359-64fd3f5f479d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:26.321192 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0dbfab-219c-4dff-a359-64fd3f5f479d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:26.333323 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 92/181] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:26 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:26.334931 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e23cb2-069b-4f44-94aa-83a64043feb9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:26.335314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e23cb2-069b-4f44-94aa-83a64043feb9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:26.335598 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e23cb2-069b-4f44-94aa-83a64043feb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:26.356503 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-51e23cb2-069b-4f44-94aa-83a64043feb9 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:26.357814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-51e23cb2-069b-4f44-94aa-83a64043feb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:26.358247 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b96705-67a1-40d9-891c-f04b2517ebcd None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:26.406457 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104d15d-5852-4b5f-aa3e-4ad868cbf72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:26.407203 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104d15d-5852-4b5f-aa3e-4ad868cbf72c None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:26.407558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104d15d-5852-4b5f-aa3e-4ad868cbf72c None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:26.407894 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104d15d-5852-4b5f-aa3e-4ad868cbf72c None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:26.408704 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1104d15d-5852-4b5f-aa3e-4ad868cbf72c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:26.409030 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1104d15d-5852-4b5f-aa3e-4ad868cbf72c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:26.410535 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1104d15d-5852-4b5f-aa3e-4ad868cbf72c None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:26.422487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 90/182] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:26 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:26.423484 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104d15d-5852-4b5f-aa3e-4ad868cbf72c None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:26.448997 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 93/183] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:26 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:26.448997 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e23cb2-069b-4f44-94aa-83a64043feb9 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:26.487265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:26.519336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:26.581076 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104d15d-5852-4b5f-aa3e-4ad868cbf72c None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:26.608709 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e23cb2-069b-4f44-94aa-83a64043feb9 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:26.628773 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa72e72-f824-4268-9b19-9c2b9ff91ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:26.629453 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa72e72-f824-4268-9b19-9c2b9ff91ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:26.629735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa72e72-f824-4268-9b19-9c2b9ff91ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:26.630011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa72e72-f824-4268-9b19-9c2b9ff91ec6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:26.630855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa72e72-f824-4268-9b19-9c2b9ff91ec6 None admin] RBAC: Adding query filter params (role.id=89b847fca3574b9083287279aee0f773, scope.project.id=6d7a7a0b84384f14849056212228badb, user.id=5b20a8c5e8314c8794af566d108815a8) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:26.631169 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa72e72-f824-4268-9b19-9c2b9ff91ec6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:26.632900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa72e72-f824-4268-9b19-9c2b9ff91ec6 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:26.640062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 91/184] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:18:26 2022] GET /identity/v3/role_assignments?user.id=5b20a8c5e8314c8794af566d108815a8&scope.project.id=6d7a7a0b84384f14849056212228badb&role.id=89b847fca3574b9083287279aee0f773 => generated 610 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:26.640990 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa72e72-f824-4268-9b19-9c2b9ff91ec6 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:26.681492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85abdcd5-013f-4618-95fd-1d27af1c92cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:26.682402 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abdcd5-013f-4618-95fd-1d27af1c92cd None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:26.682753 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abdcd5-013f-4618-95fd-1d27af1c92cd None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:26.683639 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abdcd5-013f-4618-95fd-1d27af1c92cd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:26.702259 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85abdcd5-013f-4618-95fd-1d27af1c92cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:26.712464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-85abdcd5-013f-4618-95fd-1d27af1c92cd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:18:26.712464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:26.712464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:26.712464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:26.712464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:26.712464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:26.712464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:26.712464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:26.712464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:26.712464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:26.712464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:26.712464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:26.712464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:26.712464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:26.712464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:26.712464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:26.713550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:26.713550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:26.713550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:26.713550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:26.713550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:26.713550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:26.713550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:26.713550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:26.713550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:26.713550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:26.713550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:26.713550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:26.713550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:26.713550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:26.713550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:26.713550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:26.721343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:26.721343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:26.721343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:26.721343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:26.721343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:26.721343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:26.721343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:26.721343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:26.721343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:26.721343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:26.721343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:18:26.721343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:18:26.721343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 94/185] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:26.721343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85abdcd5-013f-4618-95fd-1d27af1c92cd None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:26.832665 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:26.841387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa72e72-f824-4268-9b19-9c2b9ff91ec6 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:26.890144 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:26.899255 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85abdcd5-013f-4618-95fd-1d27af1c92cd None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:26.902661 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214687de-5f3f-45f4-a929-c64fbf331c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:26.903380 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214687de-5f3f-45f4-a929-c64fbf331c4a None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:26.903632 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214687de-5f3f-45f4-a929-c64fbf331c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:26.903910 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214687de-5f3f-45f4-a929-c64fbf331c4a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:26.908420 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214687de-5f3f-45f4-a929-c64fbf331c4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:26.908420 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214687de-5f3f-45f4-a929-c64fbf331c4a None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:26.909793 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 92/186] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:18:26 2022] GET /identity/v3/domains/default => generated 200 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:26.910047 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214687de-5f3f-45f4-a929-c64fbf331c4a None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:26.950826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43d625-dfa7-47e6-88b1-3782243621a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:26.951509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43d625-dfa7-47e6-88b1-3782243621a5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:26.951790 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43d625-dfa7-47e6-88b1-3782243621a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:26.952104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43d625-dfa7-47e6-88b1-3782243621a5 None admin] PATH_INFO: `/v3/projects/6d7a7a0b84384f14849056212228badb` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:26.953552 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff43d625-dfa7-47e6-88b1-3782243621a5 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7a7a0b84384f14849056212228badb)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:26.955039 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff43d625-dfa7-47e6-88b1-3782243621a5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:26.956947 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 95/187] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:26 2022] GET /identity/v3/projects/6d7a7a0b84384f14849056212228badb => generated 299 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:26.957686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43d625-dfa7-47e6-88b1-3782243621a5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:27.038030 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214687de-5f3f-45f4-a929-c64fbf331c4a None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:27.078244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43d625-dfa7-47e6-88b1-3782243621a5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:27.080943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e77b7d-4e71-4681-b3a8-f9f66df91af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:27.081769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e77b7d-4e71-4681-b3a8-f9f66df91af4 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:27.082077 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e77b7d-4e71-4681-b3a8-f9f66df91af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:27.082475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e77b7d-4e71-4681-b3a8-f9f66df91af4 None admin] PATH_INFO: `/v3/users/4d1eaa7a05e745a5817149d8a56dab58` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:27.083770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e77b7d-4e71-4681-b3a8-f9f66df91af4 None admin] RBAC: Authorizing `identity:get_user(user_id=4d1eaa7a05e745a5817149d8a56dab58)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:27.085415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e77b7d-4e71-4681-b3a8-f9f66df91af4 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:27.087402 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 93/188] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:26 2022] GET /identity/v3/users/4d1eaa7a05e745a5817149d8a56dab58 => generated 248 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:27.090549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e77b7d-4e71-4681-b3a8-f9f66df91af4 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:27.123715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc848e4a-6f55-4dd2-9b3a-9d519125fa3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:27.125553 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc848e4a-6f55-4dd2-9b3a-9d519125fa3e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:27.125877 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc848e4a-6f55-4dd2-9b3a-9d519125fa3e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:27.126119 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc848e4a-6f55-4dd2-9b3a-9d519125fa3e None admin] PATH_INFO: `/v3/users/778d00f8aafb4621be23568946850956` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:27.137716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:27.143519 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc848e4a-6f55-4dd2-9b3a-9d519125fa3e None admin] RBAC: Authorizing `identity:get_user(user_id=778d00f8aafb4621be23568946850956)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:27.145062 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc848e4a-6f55-4dd2-9b3a-9d519125fa3e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:27.146843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 96/189] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:26 2022] GET /identity/v3/users/778d00f8aafb4621be23568946850956 => generated 284 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:27.147683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc848e4a-6f55-4dd2-9b3a-9d519125fa3e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:27.183691 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:27.228144 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e77b7d-4e71-4681-b3a8-f9f66df91af4 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:27.265933 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd0d4c-ba03-43e3-a3ed-1a1c6eb2d37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:27.266618 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd0d4c-ba03-43e3-a3ed-1a1c6eb2d37d None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:27.266910 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd0d4c-ba03-43e3-a3ed-1a1c6eb2d37d None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:27.267184 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd0d4c-ba03-43e3-a3ed-1a1c6eb2d37d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:27.278778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd0d4c-ba03-43e3-a3ed-1a1c6eb2d37d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:27.284354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-ebdd0d4c-ba03-43e3-a3ed-1a1c6eb2d37d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:18:27.284354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:27.284354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:27.284354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:27.284354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:27.284354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:27.284354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:27.284354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:27.284354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:27.284354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:27.284354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:27.284354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:27.284354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:27.284354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:27.284354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:27.284354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:27.285493 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:27.285493 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:27.285493 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:27.285493 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:27.285493 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:27.285493 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:27.285493 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:27.285493 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:27.285493 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:27.285493 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:27.285493 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:27.285493 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:27.285493 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:27.285493 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:27.285493 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:27.285493 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:27.286505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:27.286505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:27.286505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:27.286505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:27.286505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:27.286505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:27.286505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:27.286505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:27.286505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:27.286505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:27.286505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:18:27.286505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:27.286505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 94/190] 10.222.0.121 () {56 vars in 1119 bytes} [Thu Aug 4 10:18:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:27.286505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd0d4c-ba03-43e3-a3ed-1a1c6eb2d37d None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:27.293389 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc848e4a-6f55-4dd2-9b3a-9d519125fa3e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:27.337369 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36cec5-c8c4-4510-821b-168a3115eabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:27.337985 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36cec5-c8c4-4510-821b-168a3115eabb None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:27.338294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36cec5-c8c4-4510-821b-168a3115eabb None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:27.338634 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36cec5-c8c4-4510-821b-168a3115eabb None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:27.339475 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36cec5-c8c4-4510-821b-168a3115eabb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:27.340053 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36cec5-c8c4-4510-821b-168a3115eabb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:27.341697 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36cec5-c8c4-4510-821b-168a3115eabb None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:27.351649 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 97/191] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:27 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:27.352590 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36cec5-c8c4-4510-821b-168a3115eabb None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:27.405850 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:27.414379 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd0d4c-ba03-43e3-a3ed-1a1c6eb2d37d None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:27.457565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775731a4-8868-4961-b6f7-7dd4324e669d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:27.458260 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775731a4-8868-4961-b6f7-7dd4324e669d None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:27.458617 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775731a4-8868-4961-b6f7-7dd4324e669d None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:27.458917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775731a4-8868-4961-b6f7-7dd4324e669d None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:27.459715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775731a4-8868-4961-b6f7-7dd4324e669d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:27.461089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:27.461471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775731a4-8868-4961-b6f7-7dd4324e669d None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:27.486112 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36cec5-c8c4-4510-821b-168a3115eabb None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:27.491582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 95/192] 10.222.0.121 () {60 vars in 1137 bytes} [Thu Aug 4 10:18:27 2022] POST /identity/v3/users => generated 298 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:18:27.491582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775731a4-8868-4961-b6f7-7dd4324e669d None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:27.533457 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935984b-8945-48e9-bd54-14e4fb678c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:27.534379 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935984b-8945-48e9-bd54-14e4fb678c00 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:27.534666 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935984b-8945-48e9-bd54-14e4fb678c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:27.534948 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935984b-8945-48e9-bd54-14e4fb678c00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:27.536027 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2935984b-8945-48e9-bd54-14e4fb678c00 None admin] RBAC: Adding query filter params (role.id=1b988376d12749bd8d653baa89493c5c, scope.project.id=6d7a7a0b84384f14849056212228badb, user.id=5b20a8c5e8314c8794af566d108815a8) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:27.536314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2935984b-8945-48e9-bd54-14e4fb678c00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:27.537899 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2935984b-8945-48e9-bd54-14e4fb678c00 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:27.559995 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 98/193] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:18:27 2022] GET /identity/v3/role_assignments?user.id=5b20a8c5e8314c8794af566d108815a8&scope.project.id=6d7a7a0b84384f14849056212228badb&role.id=1b988376d12749bd8d653baa89493c5c => generated 610 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:27.559995 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935984b-8945-48e9-bd54-14e4fb678c00 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:27.714713 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775731a4-8868-4961-b6f7-7dd4324e669d None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:27.782307 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2619c2b-c6a7-4d4c-a667-1832d6b3a291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:27.783896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2619c2b-c6a7-4d4c-a667-1832d6b3a291 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:27.783896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2619c2b-c6a7-4d4c-a667-1832d6b3a291 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:27.783896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2619c2b-c6a7-4d4c-a667-1832d6b3a291 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:27.795554 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935984b-8945-48e9-bd54-14e4fb678c00 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:27.813178 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:27.861356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4c991-dd1e-49c8-a4a6-10476441cf3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:27.862345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4c991-dd1e-49c8-a4a6-10476441cf3a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:27.862638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4c991-dd1e-49c8-a4a6-10476441cf3a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:27.863090 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4c991-dd1e-49c8-a4a6-10476441cf3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:27.897479 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2619c2b-c6a7-4d4c-a667-1832d6b3a291 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:27.899291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2619c2b-c6a7-4d4c-a667-1832d6b3a291 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:27.907612 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:28.001426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e4c991-dd1e-49c8-a4a6-10476441cf3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:28.002848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e4c991-dd1e-49c8-a4a6-10476441cf3a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:28.074269 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 96/194] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:18:27 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:18:28.075322 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2619c2b-c6a7-4d4c-a667-1832d6b3a291 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:28.137647 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 99/195] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:18:27 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:18:28.138510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4c991-dd1e-49c8-a4a6-10476441cf3a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:28.196422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:28.212103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:28.253738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2619c2b-c6a7-4d4c-a667-1832d6b3a291 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:28.289208 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4c991-dd1e-49c8-a4a6-10476441cf3a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:28.311835 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accbb087-4c19-4f4a-8ea0-918edf1d3de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:28.312943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbb087-4c19-4f4a-8ea0-918edf1d3de8 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:28.313259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbb087-4c19-4f4a-8ea0-918edf1d3de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:28.314165 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbb087-4c19-4f4a-8ea0-918edf1d3de8 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:28.314987 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accbb087-4c19-4f4a-8ea0-918edf1d3de8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:28.315300 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accbb087-4c19-4f4a-8ea0-918edf1d3de8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:28.317290 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accbb087-4c19-4f4a-8ea0-918edf1d3de8 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:28.333028 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 97/196] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:18:28 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:28.334203 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accbb087-4c19-4f4a-8ea0-918edf1d3de8 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:28.343612 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96ea54-80fb-401c-9f4e-acf3feb1e924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:28.344784 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96ea54-80fb-401c-9f4e-acf3feb1e924 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:28.345301 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96ea54-80fb-401c-9f4e-acf3feb1e924 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:28.345831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96ea54-80fb-401c-9f4e-acf3feb1e924 None admin] PATH_INFO: `/v3/users/d7693ebbe28d47628c3a1ec1cdff6144` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:28.347522 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96ea54-80fb-401c-9f4e-acf3feb1e924 None admin] RBAC: Authorizing `identity:get_user(user_id=d7693ebbe28d47628c3a1ec1cdff6144)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:28.349429 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96ea54-80fb-401c-9f4e-acf3feb1e924 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:28.351912 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 100/197] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:28 2022] GET /identity/v3/users/d7693ebbe28d47628c3a1ec1cdff6144 => generated 290 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:28.353265 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96ea54-80fb-401c-9f4e-acf3feb1e924 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:28.480701 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accbb087-4c19-4f4a-8ea0-918edf1d3de8 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:28.492681 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96ea54-80fb-401c-9f4e-acf3feb1e924 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:28.512357 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:28.518336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:28.530705 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbafa098-10f5-4440-954c-60a5abcf54b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:28.531390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbafa098-10f5-4440-954c-60a5abcf54b9 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:28.531680 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbafa098-10f5-4440-954c-60a5abcf54b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:28.531986 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbafa098-10f5-4440-954c-60a5abcf54b9 None admin] PATH_INFO: `/v3/projects/6d7a7a0b84384f14849056212228badb` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:28.533348 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbafa098-10f5-4440-954c-60a5abcf54b9 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7a7a0b84384f14849056212228badb)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:28.534980 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbafa098-10f5-4440-954c-60a5abcf54b9 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:28.537000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 98/198] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:28 2022] GET /identity/v3/projects/6d7a7a0b84384f14849056212228badb => generated 299 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:28.543861 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbafa098-10f5-4440-954c-60a5abcf54b9 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:28.548875 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcfb17-86d4-41fb-8ff1-058df81cc849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:28.550068 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcfb17-86d4-41fb-8ff1-058df81cc849 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:28.550658 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcfb17-86d4-41fb-8ff1-058df81cc849 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:28.551286 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcfb17-86d4-41fb-8ff1-058df81cc849 None admin] PATH_INFO: `/v3/projects/39e472078e7d42f59243230e66c387f8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:28.553145 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabcfb17-86d4-41fb-8ff1-058df81cc849 None admin] RBAC: Authorizing `identity:get_project(project_id=39e472078e7d42f59243230e66c387f8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:28.555462 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabcfb17-86d4-41fb-8ff1-058df81cc849 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:28.557984 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 101/199] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:28 2022] GET /identity/v3/projects/39e472078e7d42f59243230e66c387f8 => generated 303 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:28.559619 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcfb17-86d4-41fb-8ff1-058df81cc849 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:28.678816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbafa098-10f5-4440-954c-60a5abcf54b9 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:28.696683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcfb17-86d4-41fb-8ff1-058df81cc849 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:28.720771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e8411-3383-47ed-9122-704a254b06e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:28.721869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e8411-3383-47ed-9122-704a254b06e0 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:28.722211 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e8411-3383-47ed-9122-704a254b06e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:28.722497 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e8411-3383-47ed-9122-704a254b06e0 None admin] PATH_INFO: `/v3/users/778d00f8aafb4621be23568946850956` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:28.724231 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e8411-3383-47ed-9122-704a254b06e0 None admin] RBAC: Authorizing `identity:get_user(user_id=778d00f8aafb4621be23568946850956)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:28.727054 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e8411-3383-47ed-9122-704a254b06e0 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:28.733736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 99/200] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:28 2022] GET /identity/v3/users/778d00f8aafb4621be23568946850956 => generated 284 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:28.733736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e8411-3383-47ed-9122-704a254b06e0 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:28.751544 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ced3a-d409-4b23-86c3-aa339758dcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:28.752222 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ced3a-d409-4b23-86c3-aa339758dcee None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:28.752503 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ced3a-d409-4b23-86c3-aa339758dcee None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:28.752777 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ced3a-d409-4b23-86c3-aa339758dcee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:28.804309 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:28.822431 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:28.843435 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ced3a-d409-4b23-86c3-aa339758dcee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:28.845024 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ced3a-d409-4b23-86c3-aa339758dcee None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:28.884930 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e8411-3383-47ed-9122-704a254b06e0 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:28.935011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67968572-f02e-450d-b272-98798ab98db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:28.936526 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67968572-f02e-450d-b272-98798ab98db7 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:28.937126 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67968572-f02e-450d-b272-98798ab98db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:28.937877 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67968572-f02e-450d-b272-98798ab98db7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:28.939739 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67968572-f02e-450d-b272-98798ab98db7 None admin] RBAC: Adding query filter params (role.id=7342106a255b4bce8f150726602a8b95, scope.project.id=6d7a7a0b84384f14849056212228badb, user.id=4d1eaa7a05e745a5817149d8a56dab58) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:28.940312 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67968572-f02e-450d-b272-98798ab98db7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:28.942971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67968572-f02e-450d-b272-98798ab98db7 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:28.954010 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 100/201] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:18:28 2022] GET /identity/v3/role_assignments?user.id=4d1eaa7a05e745a5817149d8a56dab58&scope.project.id=6d7a7a0b84384f14849056212228badb&role.id=7342106a255b4bce8f150726602a8b95 => generated 610 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:28.955779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67968572-f02e-450d-b272-98798ab98db7 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:28.961837 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 102/202] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:18:28 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:18:28.962571 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ced3a-d409-4b23-86c3-aa339758dcee None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:29.078173 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:29.099552 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ced3a-d409-4b23-86c3-aa339758dcee None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:29.142878 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b820e8a-02ec-4552-8d6b-974bf2d3da2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:29.144485 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b820e8a-02ec-4552-8d6b-974bf2d3da2f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:29.144989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b820e8a-02ec-4552-8d6b-974bf2d3da2f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:29.145508 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b820e8a-02ec-4552-8d6b-974bf2d3da2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:29.196609 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67968572-f02e-450d-b272-98798ab98db7 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:29.212578 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:29.224904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b820e8a-02ec-4552-8d6b-974bf2d3da2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:29.226951 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b820e8a-02ec-4552-8d6b-974bf2d3da2f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:29.241585 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8bd48-0b0c-4744-82e6-3e7878ed7261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:29.242261 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8bd48-0b0c-4744-82e6-3e7878ed7261 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:29.242551 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8bd48-0b0c-4744-82e6-3e7878ed7261 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:29.242833 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8bd48-0b0c-4744-82e6-3e7878ed7261 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:29.243656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee8bd48-0b0c-4744-82e6-3e7878ed7261 None admin] RBAC: Adding query filter params (role.id=7342106a255b4bce8f150726602a8b95, scope.project.id=39e472078e7d42f59243230e66c387f8, user.id=778d00f8aafb4621be23568946850956) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:29.244523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee8bd48-0b0c-4744-82e6-3e7878ed7261 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:29.246558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee8bd48-0b0c-4744-82e6-3e7878ed7261 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:29.254114 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 101/203] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:18:28 2022] GET /identity/v3/role_assignments?user.id=778d00f8aafb4621be23568946850956&scope.project.id=39e472078e7d42f59243230e66c387f8&role.id=7342106a255b4bce8f150726602a8b95 => generated 260 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:29.255058 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8bd48-0b0c-4744-82e6-3e7878ed7261 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:29.400863 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8bd48-0b0c-4744-82e6-3e7878ed7261 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:29.403196 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:29.448745 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb9940-8b1d-417e-8a99-e05db5c2301c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:29.449455 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb9940-8b1d-417e-8a99-e05db5c2301c None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:29.449739 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb9940-8b1d-417e-8a99-e05db5c2301c None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:29.450018 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb9940-8b1d-417e-8a99-e05db5c2301c None admin] PATH_INFO: `/v3/projects/6d7a7a0b84384f14849056212228badb` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:29.451229 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdb9940-8b1d-417e-8a99-e05db5c2301c None admin] RBAC: Authorizing `identity:get_project(project_id=6d7a7a0b84384f14849056212228badb)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:29.452081 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 103/204] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:18:28 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:18:29.456946 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdb9940-8b1d-417e-8a99-e05db5c2301c None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:29.456946 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 102/205] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:29 2022] GET /identity/v3/projects/6d7a7a0b84384f14849056212228badb => generated 299 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:29.459598 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b820e8a-02ec-4552-8d6b-974bf2d3da2f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:29.460719 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb9940-8b1d-417e-8a99-e05db5c2301c None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:29.561593 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:29.625960 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b820e8a-02ec-4552-8d6b-974bf2d3da2f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:29.650142 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb9940-8b1d-417e-8a99-e05db5c2301c None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:29.668318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36ded0-a21d-43fb-9182-1a1159f5f82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:29.668900 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36ded0-a21d-43fb-9182-1a1159f5f82e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:29.669184 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36ded0-a21d-43fb-9182-1a1159f5f82e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:29.669460 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36ded0-a21d-43fb-9182-1a1159f5f82e None admin] PATH_INFO: `/v3/projects/39e472078e7d42f59243230e66c387f8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:29.670925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df36ded0-a21d-43fb-9182-1a1159f5f82e None admin] RBAC: Authorizing `identity:get_project(project_id=39e472078e7d42f59243230e66c387f8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:29.672458 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df36ded0-a21d-43fb-9182-1a1159f5f82e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:29.674743 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 104/206] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:29 2022] GET /identity/v3/projects/39e472078e7d42f59243230e66c387f8 => generated 303 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:29.683983 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36ded0-a21d-43fb-9182-1a1159f5f82e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:29.720274 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db7029-be26-4578-bbe5-642dc98b7ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:29.720929 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db7029-be26-4578-bbe5-642dc98b7ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:29.721214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db7029-be26-4578-bbe5-642dc98b7ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:29.721492 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db7029-be26-4578-bbe5-642dc98b7ef8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:29.723678 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db7029-be26-4578-bbe5-642dc98b7ef8 None admin] RBAC: Adding query filter params (role.id=c77f1c570e8640a3b2c836a401b91889, scope.project.id=6d7a7a0b84384f14849056212228badb, user.id=d7693ebbe28d47628c3a1ec1cdff6144) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:29.724029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db7029-be26-4578-bbe5-642dc98b7ef8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:29.725631 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db7029-be26-4578-bbe5-642dc98b7ef8 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:29.747644 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 103/207] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:18:29 2022] GET /identity/v3/role_assignments?user.id=d7693ebbe28d47628c3a1ec1cdff6144&scope.project.id=6d7a7a0b84384f14849056212228badb&role.id=c77f1c570e8640a3b2c836a401b91889 => generated 610 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:29.784026 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:29.844248 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36ded0-a21d-43fb-9182-1a1159f5f82e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:29.889598 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f5545-b4dc-4531-9c8f-b242d4ecebf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:29.890260 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f5545-b4dc-4531-9c8f-b242d4ecebf9 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:29.890544 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f5545-b4dc-4531-9c8f-b242d4ecebf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:29.890891 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f5545-b4dc-4531-9c8f-b242d4ecebf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:29.891775 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f5545-b4dc-4531-9c8f-b242d4ecebf9 None admin] RBAC: Adding query filter params (role.id=89b847fca3574b9083287279aee0f773, scope.project.id=39e472078e7d42f59243230e66c387f8, user.id=778d00f8aafb4621be23568946850956) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:29.892133 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f5545-b4dc-4531-9c8f-b242d4ecebf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:29.893697 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f5545-b4dc-4531-9c8f-b242d4ecebf9 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:29.901632 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 105/208] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:18:29 2022] GET /identity/v3/role_assignments?user.id=778d00f8aafb4621be23568946850956&scope.project.id=39e472078e7d42f59243230e66c387f8&role.id=89b847fca3574b9083287279aee0f773 => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:30.109102 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483eee8-9281-41da-8f9a-ecdb0c7f5f54 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:30.109583 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483eee8-9281-41da-8f9a-ecdb0c7f5f54 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:30.109689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483eee8-9281-41da-8f9a-ecdb0c7f5f54 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:30.111391 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 104/209] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:30.117317 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527d595-6014-4a73-9f2d-305564ffc075 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:30.117606 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527d595-6014-4a73-9f2d-305564ffc075 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:30.117866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527d595-6014-4a73-9f2d-305564ffc075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:30.140663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-3527d595-6014-4a73-9f2d-305564ffc075 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:30.141892 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3527d595-6014-4a73-9f2d-305564ffc075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:30.159903 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708b86b-4d59-4206-b138-d97673668e37 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:30.160378 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708b86b-4d59-4206-b138-d97673668e37 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:30.160666 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708b86b-4d59-4206-b138-d97673668e37 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:30.162222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 105/210] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:30.169405 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a782bf-f17d-48a3-be60-fcf67317ee76 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:30.170362 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a782bf-f17d-48a3-be60-fcf67317ee76 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:30.170880 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a782bf-f17d-48a3-be60-fcf67317ee76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:30.194900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-54a782bf-f17d-48a3-be60-fcf67317ee76 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:30.197381 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-54a782bf-f17d-48a3-be60-fcf67317ee76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:30.223556 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 106/211] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:30 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:18:30.247608 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527d595-6014-4a73-9f2d-305564ffc075 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:30.280748 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 106/212] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:30 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:18:30.300398 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:30.306286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a782bf-f17d-48a3-be60-fcf67317ee76 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:30.352665 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527d595-6014-4a73-9f2d-305564ffc075 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:30.387972 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea6896-e8cd-4e19-9d19-f0f6112645d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:30.388518 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea6896-e8cd-4e19-9d19-f0f6112645d0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:30.388794 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea6896-e8cd-4e19-9d19-f0f6112645d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:30.389064 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea6896-e8cd-4e19-9d19-f0f6112645d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:30.400301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:30.404723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea6896-e8cd-4e19-9d19-f0f6112645d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:30.411752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-23ea6896-e8cd-4e19-9d19-f0f6112645d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:30.411752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:30.411752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:30.411752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:30.411752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:30.411752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:30.411752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:30.411752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:30.411752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:30.411752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:30.411752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:30.411752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:30.411752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:30.411752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:30.411752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:30.411752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:30.412722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:30.412722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:30.412722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:30.412722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:30.412722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:30.412722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:30.412722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:30.412722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:30.412722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:30.412722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:30.412722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:30.412722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:30.412722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:30.412722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:30.412722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:30.412722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:30.413584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:30.413584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:30.413584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:30.413584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:30.413584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:30.413584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:30.413584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:30.413584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:30.413584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:30.413584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:30.413584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:30.413584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:18:30.416575 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 107/213] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:18:30.426815 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a782bf-f17d-48a3-be60-fcf67317ee76 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:30.428474 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea6896-e8cd-4e19-9d19-f0f6112645d0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:30.470791 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c39e53-c058-4af2-94f5-2b7f11dd353a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:30.471472 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c39e53-c058-4af2-94f5-2b7f11dd353a None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:30.471758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c39e53-c058-4af2-94f5-2b7f11dd353a None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:30.472064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c39e53-c058-4af2-94f5-2b7f11dd353a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:30.473347 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c39e53-c058-4af2-94f5-2b7f11dd353a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:30.474860 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c39e53-c058-4af2-94f5-2b7f11dd353a None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:30.477303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 107/214] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:18:30 2022] GET /identity/v3/domains/default => generated 200 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:18:30.482559 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c39e53-c058-4af2-94f5-2b7f11dd353a None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:30.588505 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea6896-e8cd-4e19-9d19-f0f6112645d0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:30.602403 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:30.620204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c39e53-c058-4af2-94f5-2b7f11dd353a None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:30.632966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:30.643160 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f164d-a81f-41b8-b8ff-3d5dbb8fc3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:30.643707 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f164d-a81f-41b8-b8ff-3d5dbb8fc3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:30.644024 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f164d-a81f-41b8-b8ff-3d5dbb8fc3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:30.644313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f164d-a81f-41b8-b8ff-3d5dbb8fc3c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:30.645166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f164d-a81f-41b8-b8ff-3d5dbb8fc3c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:30.645475 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f164d-a81f-41b8-b8ff-3d5dbb8fc3c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:30.647045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f164d-a81f-41b8-b8ff-3d5dbb8fc3c1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:30.663904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 108/215] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:30 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:30.667240 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f164d-a81f-41b8-b8ff-3d5dbb8fc3c1 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:30.679382 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeef3bf-dc11-49b9-bb1f-bf8597d6e88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:30.680081 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeef3bf-dc11-49b9-bb1f-bf8597d6e88a None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:30.680368 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeef3bf-dc11-49b9-bb1f-bf8597d6e88a None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:30.680644 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeef3bf-dc11-49b9-bb1f-bf8597d6e88a None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:30.682054 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeef3bf-dc11-49b9-bb1f-bf8597d6e88a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:30.683554 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeef3bf-dc11-49b9-bb1f-bf8597d6e88a None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:30.718706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 108/216] 10.222.0.121 () {60 vars in 1137 bytes} [Thu Aug 4 10:18:30 2022] POST /identity/v3/users => generated 298 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:18:30.819205 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f164d-a81f-41b8-b8ff-3d5dbb8fc3c1 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:30.860524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c344903-a080-4474-952e-1b9492e72894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:30.861261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c344903-a080-4474-952e-1b9492e72894 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:30.861550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c344903-a080-4474-952e-1b9492e72894 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:30.861850 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c344903-a080-4474-952e-1b9492e72894 None admin] PATH_INFO: `/v3/users/6ed2115c84a747d08c732316fa9089dd` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:30.879119 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c344903-a080-4474-952e-1b9492e72894 None admin] RBAC: Authorizing `identity:get_user(user_id=6ed2115c84a747d08c732316fa9089dd)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:30.880695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c344903-a080-4474-952e-1b9492e72894 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:30.882625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 109/217] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:30 2022] GET /identity/v3/users/6ed2115c84a747d08c732316fa9089dd => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:30.888691 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeef3bf-dc11-49b9-bb1f-bf8597d6e88a None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:31.004080 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeef3bf-dc11-49b9-bb1f-bf8597d6e88a None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:31.040310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13de395-4deb-45b3-934d-bcc0df1b6f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:31.040942 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13de395-4deb-45b3-934d-bcc0df1b6f9d None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:31.041219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13de395-4deb-45b3-934d-bcc0df1b6f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:31.041485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13de395-4deb-45b3-934d-bcc0df1b6f9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:31.074804 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:31.125763 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13de395-4deb-45b3-934d-bcc0df1b6f9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:31.127274 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13de395-4deb-45b3-934d-bcc0df1b6f9d None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:31.246396 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 109/218] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:18:30 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:18:31.252195 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c344903-a080-4474-952e-1b9492e72894 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:31.283464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:31.372555 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c344903-a080-4474-952e-1b9492e72894 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:31.407915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e3c693-327e-4bfb-a15b-3098bb1e03b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:31.408607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e3c693-327e-4bfb-a15b-3098bb1e03b6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:31.408893 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e3c693-327e-4bfb-a15b-3098bb1e03b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:31.409171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e3c693-327e-4bfb-a15b-3098bb1e03b6 None admin] PATH_INFO: `/v3/projects/39e472078e7d42f59243230e66c387f8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:31.410387 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e3c693-327e-4bfb-a15b-3098bb1e03b6 None admin] RBAC: Authorizing `identity:get_project(project_id=39e472078e7d42f59243230e66c387f8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:31.411874 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e3c693-327e-4bfb-a15b-3098bb1e03b6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:31.413905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 110/219] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:31 2022] GET /identity/v3/projects/39e472078e7d42f59243230e66c387f8 => generated 303 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:31.419872 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13de395-4deb-45b3-934d-bcc0df1b6f9d None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:31.487613 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:31.529797 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13de395-4deb-45b3-934d-bcc0df1b6f9d None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:31.567143 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f376b3a-5385-4fbe-b0df-12826bdeaaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:31.567922 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f376b3a-5385-4fbe-b0df-12826bdeaaf8 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:31.568282 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f376b3a-5385-4fbe-b0df-12826bdeaaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:31.568591 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f376b3a-5385-4fbe-b0df-12826bdeaaf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:31.569496 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f376b3a-5385-4fbe-b0df-12826bdeaaf8 None admin] RBAC: Adding query filter params (role.id=1b988376d12749bd8d653baa89493c5c, scope.project.id=39e472078e7d42f59243230e66c387f8, user.id=6ed2115c84a747d08c732316fa9089dd) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:31.569844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f376b3a-5385-4fbe-b0df-12826bdeaaf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:31.571623 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f376b3a-5385-4fbe-b0df-12826bdeaaf8 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:31.579741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 110/220] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:18:31 2022] GET /identity/v3/role_assignments?user.id=6ed2115c84a747d08c732316fa9089dd&scope.project.id=39e472078e7d42f59243230e66c387f8&role.id=1b988376d12749bd8d653baa89493c5c => generated 260 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:31.932283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ded61-80c1-4401-9864-c0bb91cd8b04 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:31.932486 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ded61-80c1-4401-9864-c0bb91cd8b04 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:31.932747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ded61-80c1-4401-9864-c0bb91cd8b04 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:31.934548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 111/221] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:31.940637 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9236a-9398-4237-85b8-b41d94ff0d02 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:31.940957 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9236a-9398-4237-85b8-b41d94ff0d02 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:31.941209 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9236a-9398-4237-85b8-b41d94ff0d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:31.963441 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-d1b9236a-9398-4237-85b8-b41d94ff0d02 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:31.964811 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d1b9236a-9398-4237-85b8-b41d94ff0d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:32.073792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 111/222] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:31 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:32.106898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ded61-80c1-4401-9864-c0bb91cd8b04 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:32.129582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ffbe8-28f0-4bb0-b4af-69ca74f4fc91 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:32.129875 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ffbe8-28f0-4bb0-b4af-69ca74f4fc91 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:32.130160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ffbe8-28f0-4bb0-b4af-69ca74f4fc91 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:32.131690 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 112/223] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:32.138208 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cab5d7-99e0-4da8-804d-37c709ffbd07 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:32.138509 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cab5d7-99e0-4da8-804d-37c709ffbd07 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:32.138747 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cab5d7-99e0-4da8-804d-37c709ffbd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:32.158012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-70cab5d7-99e0-4da8-804d-37c709ffbd07 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:32.159326 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-70cab5d7-99e0-4da8-804d-37c709ffbd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:32.224440 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:32.240047 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 113/224] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:32 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:18:32.266860 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cab5d7-99e0-4da8-804d-37c709ffbd07 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:32.321399 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:32.343801 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ded61-80c1-4401-9864-c0bb91cd8b04 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:32.389382 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cab5d7-99e0-4da8-804d-37c709ffbd07 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:32.432817 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a637017-a709-4eb2-9099-cb236bf058d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:32.433249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948191d-5b24-4188-9668-ca686a343e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:32.433505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a637017-a709-4eb2-9099-cb236bf058d4 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:32.433789 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a637017-a709-4eb2-9099-cb236bf058d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:32.433889 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948191d-5b24-4188-9668-ca686a343e44 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:32.434063 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a637017-a709-4eb2-9099-cb236bf058d4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:32.434157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948191d-5b24-4188-9668-ca686a343e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:32.434438 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948191d-5b24-4188-9668-ca686a343e44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:32.446954 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a637017-a709-4eb2-9099-cb236bf058d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:32.452517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-7a637017-a709-4eb2-9099-cb236bf058d4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:18:32.452517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:32.452517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:32.452517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:32.452517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:32.452517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:32.452517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:32.452517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:32.452517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:32.452517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:32.452517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:32.452517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:32.452517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:32.452517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:32.452517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:32.452517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:32.453500 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:32.453500 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:32.453500 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:32.453500 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:32.453500 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:32.453500 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:32.453500 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:32.453500 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:32.453500 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:32.453500 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:32.453500 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:32.453500 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:32.453500 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:32.453500 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:32.453500 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:32.453500 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:32.454320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:32.454320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:32.454320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:32.454320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:32.454320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:32.454320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:32.454320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:32.454320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:32.454320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:32.454320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:32.454320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:18:32.454320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:32.454320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 114/225] 10.222.0.121 () {56 vars in 1119 bytes} [Thu Aug 4 10:18:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:18:32.460776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a637017-a709-4eb2-9099-cb236bf058d4 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:32.480797 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948191d-5b24-4188-9668-ca686a343e44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:32.485747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-4948191d-5b24-4188-9668-ca686a343e44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:18:32.485747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:32.485747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:32.485747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:32.485747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:32.485747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:32.485747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:32.485747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:32.485747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:32.485747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:32.485747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:32.485747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:32.485747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:32.485747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:32.485747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:32.485747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:32.488288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:32.488288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:32.488288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:32.488288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:32.488288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:32.488288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:32.488288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:32.488288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:32.488288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:32.488288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:32.488288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:32.488288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:32.488288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:32.488288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:32.488288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:32.488288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:32.490663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:32.490663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:32.490663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:32.490663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:32.490663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:32.490663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:32.490663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:32.490663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:32.490663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:32.490663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:32.490663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:18:32.490663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:18:32.490663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 112/226] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:18:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:32.501047 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948191d-5b24-4188-9668-ca686a343e44 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:32.520416 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:32.630025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a637017-a709-4eb2-9099-cb236bf058d4 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:32.722836 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75931b-ccef-41df-94e1-b268d5871df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:32.723511 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75931b-ccef-41df-94e1-b268d5871df9 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:32.723800 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75931b-ccef-41df-94e1-b268d5871df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:32.724109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75931b-ccef-41df-94e1-b268d5871df9 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:32.724880 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75931b-ccef-41df-94e1-b268d5871df9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:32.725199 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75931b-ccef-41df-94e1-b268d5871df9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:32.726738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75931b-ccef-41df-94e1-b268d5871df9 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:32.750915 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 115/227] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:18:32 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:18:32.757364 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75931b-ccef-41df-94e1-b268d5871df9 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:32.773252 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948191d-5b24-4188-9668-ca686a343e44 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:32.828859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9532b-5fe0-4ad8-9d31-f1658329e054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:32.829524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9532b-5fe0-4ad8-9d31-f1658329e054 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:32.829814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9532b-5fe0-4ad8-9d31-f1658329e054 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:32.830096 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9532b-5fe0-4ad8-9d31-f1658329e054 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:32.830857 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9532b-5fe0-4ad8-9d31-f1658329e054 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:32.831172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9532b-5fe0-4ad8-9d31-f1658329e054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:32.832693 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9532b-5fe0-4ad8-9d31-f1658329e054 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:32.854659 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 113/228] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:32 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:32.860442 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9532b-5fe0-4ad8-9d31-f1658329e054 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:32.898175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75931b-ccef-41df-94e1-b268d5871df9 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:32.908418 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:32.909279 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:32.949849 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15735b1-120d-4e57-868d-0512a8e20a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:32.950548 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15735b1-120d-4e57-868d-0512a8e20a46 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:32.950827 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15735b1-120d-4e57-868d-0512a8e20a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:32.951103 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15735b1-120d-4e57-868d-0512a8e20a46 None admin] PATH_INFO: `/v3/users/778d00f8aafb4621be23568946850956` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:32.952413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15735b1-120d-4e57-868d-0512a8e20a46 None admin] RBAC: Authorizing `identity:get_user(user_id=778d00f8aafb4621be23568946850956)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:32.954258 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15735b1-120d-4e57-868d-0512a8e20a46 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:32.956466 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 116/229] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:32 2022] GET /identity/v3/users/778d00f8aafb4621be23568946850956 => generated 284 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:32.965387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15735b1-120d-4e57-868d-0512a8e20a46 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:33.097825 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15735b1-120d-4e57-868d-0512a8e20a46 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:33.138743 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9532b-5fe0-4ad8-9d31-f1658329e054 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:33.149836 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf970620-9cca-461f-9c05-0b1517a6a84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:33.150538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf970620-9cca-461f-9c05-0b1517a6a84a None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:33.150820 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf970620-9cca-461f-9c05-0b1517a6a84a None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:33.151197 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf970620-9cca-461f-9c05-0b1517a6a84a None admin] PATH_INFO: `/v3/projects/39e472078e7d42f59243230e66c387f8` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:33.153299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf970620-9cca-461f-9c05-0b1517a6a84a None admin] RBAC: Authorizing `identity:get_project(project_id=39e472078e7d42f59243230e66c387f8)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:33.154829 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf970620-9cca-461f-9c05-0b1517a6a84a None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:33.157168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 117/230] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:32 2022] GET /identity/v3/projects/39e472078e7d42f59243230e66c387f8 => generated 303 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:33.164206 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf970620-9cca-461f-9c05-0b1517a6a84a None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:33.196770 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322b805-348c-461d-b805-9c27c2fd6171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:33.197452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322b805-348c-461d-b805-9c27c2fd6171 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:33.197738 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322b805-348c-461d-b805-9c27c2fd6171 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:33.198018 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322b805-348c-461d-b805-9c27c2fd6171 None admin] PATH_INFO: `/v3/users/778d00f8aafb4621be23568946850956` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:33.200717 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322b805-348c-461d-b805-9c27c2fd6171 None admin] RBAC: Authorizing `identity:get_user(user_id=778d00f8aafb4621be23568946850956)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:33.202236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322b805-348c-461d-b805-9c27c2fd6171 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:33.212573 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 114/231] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:32 2022] GET /identity/v3/users/778d00f8aafb4621be23568946850956 => generated 284 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:33.215212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322b805-348c-461d-b805-9c27c2fd6171 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:33.265434 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:33.366501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf970620-9cca-461f-9c05-0b1517a6a84a None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:33.378028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:33.402196 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322b805-348c-461d-b805-9c27c2fd6171 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:33.428822 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e367493-ec10-4ba9-877c-b24cb093fa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:33.429536 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e367493-ec10-4ba9-877c-b24cb093fa99 None admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:33.429825 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e367493-ec10-4ba9-877c-b24cb093fa99 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:33.430107 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e367493-ec10-4ba9-877c-b24cb093fa99 None admin] PATH_INFO: `/v3/projects/39e472078e7d42f59243230e66c387f8/users/778d00f8aafb4621be23568946850956/roles/89b847fca3574b9083287279aee0f773` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:33.433935 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e367493-ec10-4ba9-877c-b24cb093fa99 None admin] RBAC: Authorizing `identity:create_grant(project_id=39e472078e7d42f59243230e66c387f8, user_id=778d00f8aafb4621be23568946850956, role_id=89b847fca3574b9083287279aee0f773)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:33.435445 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e367493-ec10-4ba9-877c-b24cb093fa99 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:33.449409 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17015a-d300-4032-9f08-4216bb1b043c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:33.450115 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17015a-d300-4032-9f08-4216bb1b043c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:33.450498 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17015a-d300-4032-9f08-4216bb1b043c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:33.450782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17015a-d300-4032-9f08-4216bb1b043c None admin] PATH_INFO: `/v3/projects/39e472078e7d42f59243230e66c387f8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:33.455096 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17015a-d300-4032-9f08-4216bb1b043c None admin] RBAC: Authorizing `identity:get_project(project_id=39e472078e7d42f59243230e66c387f8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:33.457723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17015a-d300-4032-9f08-4216bb1b043c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:33.461708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 118/232] 10.222.0.121 () {58 vars in 1444 bytes} [Thu Aug 4 10:18:33 2022] PUT /identity/v3/projects/39e472078e7d42f59243230e66c387f8/users/778d00f8aafb4621be23568946850956/roles/89b847fca3574b9083287279aee0f773 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:18:33.462792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 115/233] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:33 2022] GET /identity/v3/projects/39e472078e7d42f59243230e66c387f8 => generated 303 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:18:33.474385 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e367493-ec10-4ba9-877c-b24cb093fa99 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:33.616958 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b193d-df67-438f-b11e-84b9fff999bd None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:33.617304 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b193d-df67-438f-b11e-84b9fff999bd None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:33.617628 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b193d-df67-438f-b11e-84b9fff999bd None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:33.619399 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 116/234] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:33 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:33.624120 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b3174-82fd-4b70-aff2-7d0f66b42380 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:33.624431 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b3174-82fd-4b70-aff2-7d0f66b42380 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:33.624726 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b3174-82fd-4b70-aff2-7d0f66b42380 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:33.626674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 117/235] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:33 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 10:18:33.627824 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c13ecf-23d5-4ab6-913c-4abdf439f836 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:33.628149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c13ecf-23d5-4ab6-913c-4abdf439f836 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:33.628438 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c13ecf-23d5-4ab6-913c-4abdf439f836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:33.656533 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b5c13ecf-23d5-4ab6-913c-4abdf439f836 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:33.657821 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b5c13ecf-23d5-4ab6-913c-4abdf439f836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:33.663508 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e367493-ec10-4ba9-877c-b24cb093fa99 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:33.667092 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:33.721339 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8cdb5-3e0d-4e8f-8c10-ca263dc785d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:33.722374 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8cdb5-3e0d-4e8f-8c10-ca263dc785d0 None admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:33.722663 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8cdb5-3e0d-4e8f-8c10-ca263dc785d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:33.722948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8cdb5-3e0d-4e8f-8c10-ca263dc785d0 None admin] PATH_INFO: `/v3/projects/39e472078e7d42f59243230e66c387f8/users/778d00f8aafb4621be23568946850956/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:33.726335 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b8cdb5-3e0d-4e8f-8c10-ca263dc785d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=39e472078e7d42f59243230e66c387f8, user_id=778d00f8aafb4621be23568946850956, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:33.728240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b8cdb5-3e0d-4e8f-8c10-ca263dc785d0 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:33.752039 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 119/236] 10.222.0.121 () {58 vars in 1444 bytes} [Thu Aug 4 10:18:33 2022] PUT /identity/v3/projects/39e472078e7d42f59243230e66c387f8/users/778d00f8aafb4621be23568946850956/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:18:33.753807 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b36839-599f-44be-b8b4-b82bc2a9364f None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:33.754101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b36839-599f-44be-b8b4-b82bc2a9364f None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:33.754383 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b36839-599f-44be-b8b4-b82bc2a9364f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:33.755047 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 118/237] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:33 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:33.775677 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-74b36839-599f-44be-b8b4-b82bc2a9364f None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:33.777109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-74b36839-599f-44be-b8b4-b82bc2a9364f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:33.778857 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c13ecf-23d5-4ab6-913c-4abdf439f836 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:33.885817 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 120/238] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:33 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:33.894088 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:33.915555 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c13ecf-23d5-4ab6-913c-4abdf439f836 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:33.915785 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b36839-599f-44be-b8b4-b82bc2a9364f None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:33.958242 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b0d6a-16ed-4aaf-aade-4d0858af1e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:33.958903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b0d6a-16ed-4aaf-aade-4d0858af1e37 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:33.959184 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b0d6a-16ed-4aaf-aade-4d0858af1e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:33.959459 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b0d6a-16ed-4aaf-aade-4d0858af1e37 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:33.974439 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b0d6a-16ed-4aaf-aade-4d0858af1e37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:33.980149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-d04b0d6a-16ed-4aaf-aade-4d0858af1e37 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:18:33.980149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:33.980149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:33.980149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:33.980149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:33.980149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:33.980149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:33.980149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:33.980149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:33.980149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:33.980149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:33.980149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:33.980149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:33.980149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:33.980149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:33.980149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:33.981103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:33.981103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:33.981103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:33.981103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:33.981103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:33.981103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:33.981103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:33.981103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:33.981103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:33.981103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:33.981103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:33.981103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:33.981103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:33.981103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:33.981103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:33.981103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:33.981916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:33.981916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:33.981916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:33.981916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:33.981916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:33.981916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:33.981916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:33.981916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:33.981916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:33.981916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:33.981916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:18:33.981916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:18:33.981916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 119/239] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:18:33.987773 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b0d6a-16ed-4aaf-aade-4d0858af1e37 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:34.031807 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:34.044791 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b36839-599f-44be-b8b4-b82bc2a9364f None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:34.088242 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc147bea-e02b-4dfd-94b2-bbe2e92bec5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:34.088964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc147bea-e02b-4dfd-94b2-bbe2e92bec5e None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:34.089249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc147bea-e02b-4dfd-94b2-bbe2e92bec5e None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:34.089845 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc147bea-e02b-4dfd-94b2-bbe2e92bec5e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:34.091623 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc147bea-e02b-4dfd-94b2-bbe2e92bec5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:34.093231 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc147bea-e02b-4dfd-94b2-bbe2e92bec5e None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:34.095252 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 121/240] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:18:33 2022] GET /identity/v3/domains/default => generated 200 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:18:34.100763 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc147bea-e02b-4dfd-94b2-bbe2e92bec5e None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:34.214355 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b0d6a-16ed-4aaf-aade-4d0858af1e37 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:34.225149 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc147bea-e02b-4dfd-94b2-bbe2e92bec5e None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:34.266895 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad821a3a-5740-449f-ba74-e64b54551753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:34.267654 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad821a3a-5740-449f-ba74-e64b54551753 None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:34.268071 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad821a3a-5740-449f-ba74-e64b54551753 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:34.268473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad821a3a-5740-449f-ba74-e64b54551753 None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:34.271971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad821a3a-5740-449f-ba74-e64b54551753 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:34.271971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad821a3a-5740-449f-ba74-e64b54551753 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:34.301041 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 122/241] 10.222.0.121 () {60 vars in 1137 bytes} [Thu Aug 4 10:18:34 2022] POST /identity/v3/users => generated 294 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:18:34.303163 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde9c73-d1ee-477a-bf82-58f5ac40ba9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:34.304221 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde9c73-d1ee-477a-bf82-58f5ac40ba9c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:34.304708 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde9c73-d1ee-477a-bf82-58f5ac40ba9c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:34.305193 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde9c73-d1ee-477a-bf82-58f5ac40ba9c None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:34.306148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde9c73-d1ee-477a-bf82-58f5ac40ba9c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:34.306646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde9c73-d1ee-477a-bf82-58f5ac40ba9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:34.308358 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde9c73-d1ee-477a-bf82-58f5ac40ba9c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:34.326996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 120/242] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:33 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:18:34.333026 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad821a3a-5740-449f-ba74-e64b54551753 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:34.356882 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df41fd-5afe-4cd6-9aa6-ce3538cd07cc None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:34.357797 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df41fd-5afe-4cd6-9aa6-ce3538cd07cc None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:34.358258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df41fd-5afe-4cd6-9aa6-ce3538cd07cc None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:34.360931 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 121/243] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:34 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:34.369919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc392436-c9ba-4497-a54d-88005efae4fa None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:34.370510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc392436-c9ba-4497-a54d-88005efae4fa None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:34.370792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc392436-c9ba-4497-a54d-88005efae4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:34.376250 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:34.389055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:34.413181 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-dc392436-c9ba-4497-a54d-88005efae4fa None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:34.414448 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-dc392436-c9ba-4497-a54d-88005efae4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:34.512722 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad821a3a-5740-449f-ba74-e64b54551753 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:34.562158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 122/244] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:34 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:34.569685 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5bb01-98fc-4a89-b90f-e8ab88a2ae0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:34.570440 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5bb01-98fc-4a89-b90f-e8ab88a2ae0d None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:34.570958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5bb01-98fc-4a89-b90f-e8ab88a2ae0d None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:34.571096 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5bb01-98fc-4a89-b90f-e8ab88a2ae0d None admin] PATH_INFO: `/v3/users/b21d239600dc4423a7c794c39d189104` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:34.599332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5bb01-98fc-4a89-b90f-e8ab88a2ae0d None admin] RBAC: Authorizing `identity:get_user(user_id=b21d239600dc4423a7c794c39d189104)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:34.601142 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5bb01-98fc-4a89-b90f-e8ab88a2ae0d None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:34.603326 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 123/245] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:34 2022] GET /identity/v3/users/b21d239600dc4423a7c794c39d189104 => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:34.604243 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc392436-c9ba-4497-a54d-88005efae4fa None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:34.610863 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5bb01-98fc-4a89-b90f-e8ab88a2ae0d None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:34.722621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:34.754663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc392436-c9ba-4497-a54d-88005efae4fa None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:34.762388 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5bb01-98fc-4a89-b90f-e8ab88a2ae0d None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:34.806564 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f0bd3-3410-4673-b989-cc590cacc707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:34.807552 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f0bd3-3410-4673-b989-cc590cacc707 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:34.808035 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f0bd3-3410-4673-b989-cc590cacc707 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:34.808438 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f0bd3-3410-4673-b989-cc590cacc707 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:34.810776 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:34.816783 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada5d41-c23a-4f2b-815c-45aba6872d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:34.817583 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada5d41-c23a-4f2b-815c-45aba6872d42 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:34.817965 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada5d41-c23a-4f2b-815c-45aba6872d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:34.819306 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada5d41-c23a-4f2b-815c-45aba6872d42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:34.826025 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f0bd3-3410-4673-b989-cc590cacc707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:34.831236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-244f0bd3-3410-4673-b989-cc590cacc707 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:34.831236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:34.831236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:34.831236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:34.831236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:34.831236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:34.831236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:34.831236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:34.831236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:34.831236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:34.831236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:34.831236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:34.831236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:34.831236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:34.831236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:34.831236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:34.832338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:34.832338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:34.832338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:34.832338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:34.832338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:34.832338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:34.832338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:34.832338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:34.832338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:34.832338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:34.832338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:34.832338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:34.832338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:34.832338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:34.832338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:34.832338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:34.833249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:34.833249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:34.833249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:34.833249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:34.833249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:34.833249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:34.833249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:34.833249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:34.833249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:34.833249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:34.833249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:34.833249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:18:34.835520 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 123/246] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:34.841562 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f0bd3-3410-4673-b989-cc590cacc707 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:34.912367 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada5d41-c23a-4f2b-815c-45aba6872d42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:34.913969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada5d41-c23a-4f2b-815c-45aba6872d42 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:34.968695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f0bd3-3410-4673-b989-cc590cacc707 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:35.009456 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b65a2-dea5-449d-b46d-b9b0733cbef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:35.010221 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b65a2-dea5-449d-b46d-b9b0733cbef6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:35.010562 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b65a2-dea5-449d-b46d-b9b0733cbef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:35.010979 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b65a2-dea5-449d-b46d-b9b0733cbef6 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:35.011784 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b65a2-dea5-449d-b46d-b9b0733cbef6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:35.012187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b65a2-dea5-449d-b46d-b9b0733cbef6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:35.013707 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b65a2-dea5-449d-b46d-b9b0733cbef6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:35.017324 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:35.026455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 124/247] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:34 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:18:35.033066 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b65a2-dea5-449d-b46d-b9b0733cbef6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:35.058812 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 124/248] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:18:34 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 10:18:35.065201 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada5d41-c23a-4f2b-815c-45aba6872d42 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:35.102131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:35.151974 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b65a2-dea5-449d-b46d-b9b0733cbef6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:35.189294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184cd768-fa7f-4745-a74c-2dac02151ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:35.189972 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184cd768-fa7f-4745-a74c-2dac02151ffa None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:35.190253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184cd768-fa7f-4745-a74c-2dac02151ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:35.190527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184cd768-fa7f-4745-a74c-2dac02151ffa None admin] PATH_INFO: `/v3/users/6ed2115c84a747d08c732316fa9089dd` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:35.191985 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184cd768-fa7f-4745-a74c-2dac02151ffa None admin] RBAC: Authorizing `identity:get_user(user_id=6ed2115c84a747d08c732316fa9089dd)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:35.193773 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184cd768-fa7f-4745-a74c-2dac02151ffa None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:35.195795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 125/249] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:35 2022] GET /identity/v3/users/6ed2115c84a747d08c732316fa9089dd => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:18:35.201885 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184cd768-fa7f-4745-a74c-2dac02151ffa None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:35.298233 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada5d41-c23a-4f2b-815c-45aba6872d42 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:35.310348 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184cd768-fa7f-4745-a74c-2dac02151ffa None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:35.339135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:35.351685 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc51f8c-d820-4490-9c99-811b7cfc14f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:35.352771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc51f8c-d820-4490-9c99-811b7cfc14f5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:35.353201 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc51f8c-d820-4490-9c99-811b7cfc14f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:35.353620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc51f8c-d820-4490-9c99-811b7cfc14f5 None admin] PATH_INFO: `/v3/projects/39e472078e7d42f59243230e66c387f8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:35.355065 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc51f8c-d820-4490-9c99-811b7cfc14f5 None admin] RBAC: Authorizing `identity:get_project(project_id=39e472078e7d42f59243230e66c387f8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:35.356797 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc51f8c-d820-4490-9c99-811b7cfc14f5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:35.358689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 126/250] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:35 2022] GET /identity/v3/projects/39e472078e7d42f59243230e66c387f8 => generated 303 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:35.365333 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc51f8c-d820-4490-9c99-811b7cfc14f5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:35.423301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36651e9-f2f3-42ea-8d1c-739caa192e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:35.424047 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36651e9-f2f3-42ea-8d1c-739caa192e48 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:35.424324 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36651e9-f2f3-42ea-8d1c-739caa192e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:35.425067 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36651e9-f2f3-42ea-8d1c-739caa192e48 None admin] PATH_INFO: `/v3/projects/39e472078e7d42f59243230e66c387f8` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:35.428762 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36651e9-f2f3-42ea-8d1c-739caa192e48 None admin] RBAC: Authorizing `identity:get_project(project_id=39e472078e7d42f59243230e66c387f8)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:35.430311 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36651e9-f2f3-42ea-8d1c-739caa192e48 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:35.432521 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 125/251] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:35 2022] GET /identity/v3/projects/39e472078e7d42f59243230e66c387f8 => generated 303 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:35.438145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36651e9-f2f3-42ea-8d1c-739caa192e48 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:35.488865 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc51f8c-d820-4490-9c99-811b7cfc14f5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:35.525709 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:35.529246 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3ed96-5a8a-4a22-afad-cded147bc8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:35.530127 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3ed96-5a8a-4a22-afad-cded147bc8b1 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:35.530711 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3ed96-5a8a-4a22-afad-cded147bc8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:35.532073 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3ed96-5a8a-4a22-afad-cded147bc8b1 None admin] PATH_INFO: `/v3/projects/39e472078e7d42f59243230e66c387f8/users/6ed2115c84a747d08c732316fa9089dd/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:35.534465 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3ed96-5a8a-4a22-afad-cded147bc8b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=39e472078e7d42f59243230e66c387f8, user_id=6ed2115c84a747d08c732316fa9089dd, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:35.536219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3ed96-5a8a-4a22-afad-cded147bc8b1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:35.555493 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 127/252] 10.222.0.121 () {58 vars in 1444 bytes} [Thu Aug 4 10:18:35 2022] PUT /identity/v3/projects/39e472078e7d42f59243230e66c387f8/users/6ed2115c84a747d08c732316fa9089dd/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:18:35.625729 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36651e9-f2f3-42ea-8d1c-739caa192e48 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:35.675063 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d52bd1-2f47-4458-83ae-f897b0bb9a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:35.676109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d52bd1-2f47-4458-83ae-f897b0bb9a42 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:35.676509 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d52bd1-2f47-4458-83ae-f897b0bb9a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:35.676870 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d52bd1-2f47-4458-83ae-f897b0bb9a42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:35.677751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d52bd1-2f47-4458-83ae-f897b0bb9a42 None admin] RBAC: Adding query filter params (role.id=c77f1c570e8640a3b2c836a401b91889, scope.project.id=39e472078e7d42f59243230e66c387f8, user.id=b21d239600dc4423a7c794c39d189104) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:35.678073 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d52bd1-2f47-4458-83ae-f897b0bb9a42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:35.680035 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d52bd1-2f47-4458-83ae-f897b0bb9a42 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:35.688319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 126/253] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:18:35 2022] GET /identity/v3/role_assignments?user.id=b21d239600dc4423a7c794c39d189104&scope.project.id=39e472078e7d42f59243230e66c387f8&role.id=c77f1c570e8640a3b2c836a401b91889 => generated 260 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:36.160664 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d008dd-eb15-454b-b712-126cb5c392e6 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:36.160928 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d008dd-eb15-454b-b712-126cb5c392e6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:36.161249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d008dd-eb15-454b-b712-126cb5c392e6 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:36.162990 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 128/254] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:36.168829 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c09a3b-cb81-45ef-b462-b6187e92d1cb None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:36.169118 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c09a3b-cb81-45ef-b462-b6187e92d1cb None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:36.169397 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c09a3b-cb81-45ef-b462-b6187e92d1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:36.188727 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-e8c09a3b-cb81-45ef-b462-b6187e92d1cb None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:36.189958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-e8c09a3b-cb81-45ef-b462-b6187e92d1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:36.278661 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 127/255] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:36 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:18:36.326899 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d008dd-eb15-454b-b712-126cb5c392e6 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:36.404112 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:36.469458 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d008dd-eb15-454b-b712-126cb5c392e6 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:36.515563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9113e3-67b0-4388-b47d-357df2e933d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:36.516301 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9113e3-67b0-4388-b47d-357df2e933d2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:36.516571 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9113e3-67b0-4388-b47d-357df2e933d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:36.516796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9113e3-67b0-4388-b47d-357df2e933d2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:36.534026 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9113e3-67b0-4388-b47d-357df2e933d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:36.539620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-9b9113e3-67b0-4388-b47d-357df2e933d2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:18:36.539620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:36.539620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:36.539620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:36.539620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:36.539620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:36.539620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:36.539620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:36.539620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:36.539620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:36.539620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:36.539620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:36.539620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:36.539620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:36.539620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:36.539620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:36.540615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:36.540615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:36.540615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:36.540615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:36.540615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:36.540615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:36.540615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:36.540615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:36.540615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:36.540615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:36.540615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:36.540615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:36.540615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:36.540615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:36.540615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:36.540615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:36.541440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:36.541440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:36.541440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:36.541440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:36.541440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:36.541440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:36.541440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:36.541440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:36.541440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:36.541440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:36.541440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:18:36.541440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:18:36.541440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 129/256] 10.222.0.121 () {56 vars in 1119 bytes} [Thu Aug 4 10:18:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:36.547303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c09a3b-cb81-45ef-b462-b6187e92d1cb None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:36.629102 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:36.745265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c09a3b-cb81-45ef-b462-b6187e92d1cb None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:36.778322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b6a41-3621-47ab-affe-3fcf230916be None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:36.778601 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b6a41-3621-47ab-affe-3fcf230916be None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:36.778879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b6a41-3621-47ab-affe-3fcf230916be None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:36.783402 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 130/257] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:36.786468 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de8b8f-6310-4473-a2ff-4124d8b031e2 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:36.786763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de8b8f-6310-4473-a2ff-4124d8b031e2 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:36.787019 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de8b8f-6310-4473-a2ff-4124d8b031e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:36.806577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1297bec-a06e-44c0-a8b0-75a4d49883e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:36.807698 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1297bec-a06e-44c0-a8b0-75a4d49883e7 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:36.808256 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1297bec-a06e-44c0-a8b0-75a4d49883e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:36.808720 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1297bec-a06e-44c0-a8b0-75a4d49883e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:36.809725 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1297bec-a06e-44c0-a8b0-75a4d49883e7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:36.810232 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1297bec-a06e-44c0-a8b0-75a4d49883e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:36.811889 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1297bec-a06e-44c0-a8b0-75a4d49883e7 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:36.825345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f1de8b8f-6310-4473-a2ff-4124d8b031e2 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:36.826585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f1de8b8f-6310-4473-a2ff-4124d8b031e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:36.830055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 128/258] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:18:36 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:36.836425 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1297bec-a06e-44c0-a8b0-75a4d49883e7 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:36.981753 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 131/259] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:36 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:18:37.000400 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:37.008378 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1297bec-a06e-44c0-a8b0-75a4d49883e7 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:37.055429 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de8b8f-6310-4473-a2ff-4124d8b031e2 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:37.059118 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e0f45-2be4-49a4-bf8c-020999d57e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:37.059816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e0f45-2be4-49a4-bf8c-020999d57e52 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:37.060131 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e0f45-2be4-49a4-bf8c-020999d57e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:37.060412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e0f45-2be4-49a4-bf8c-020999d57e52 None admin] PATH_INFO: `/v3/users/778d00f8aafb4621be23568946850956` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:37.061770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e0f45-2be4-49a4-bf8c-020999d57e52 None admin] RBAC: Authorizing `identity:get_user(user_id=778d00f8aafb4621be23568946850956)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:37.063305 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e0f45-2be4-49a4-bf8c-020999d57e52 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:37.065410 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 129/260] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:36 2022] GET /identity/v3/users/778d00f8aafb4621be23568946850956 => generated 284 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:18:37.071121 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e0f45-2be4-49a4-bf8c-020999d57e52 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:37.158641 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:37.219913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de8b8f-6310-4473-a2ff-4124d8b031e2 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:37.258682 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2503d8d0-91df-4e69-8a38-de8467e66bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:37.259245 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503d8d0-91df-4e69-8a38-de8467e66bae None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:37.259524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503d8d0-91df-4e69-8a38-de8467e66bae None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:37.259799 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503d8d0-91df-4e69-8a38-de8467e66bae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:37.264706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e0f45-2be4-49a4-bf8c-020999d57e52 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:37.274404 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2503d8d0-91df-4e69-8a38-de8467e66bae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:37.281922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-2503d8d0-91df-4e69-8a38-de8467e66bae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:18:37.281922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:37.281922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:37.281922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:37.281922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:37.281922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:37.281922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:37.281922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:37.281922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:37.281922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:37.281922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:37.281922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:37.281922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:37.281922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:37.281922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:37.281922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:37.283207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:37.283207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:37.283207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:37.283207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:37.283207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:37.283207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:37.283207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:37.283207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:37.283207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:37.283207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:37.283207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:37.283207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:37.283207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:37.283207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:37.283207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:37.283207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:37.284082 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:37.284082 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:37.284082 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:37.284082 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:37.284082 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:37.284082 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:37.284082 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:37.284082 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:37.284082 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:37.284082 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:37.284082 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:18:37.284082 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:18:37.284082 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 132/261] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:18:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:37.290332 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2503d8d0-91df-4e69-8a38-de8467e66bae None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:37.337648 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f3701e-d09a-4af5-85ff-24816d072d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:37.338663 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3701e-d09a-4af5-85ff-24816d072d80 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:37.339248 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3701e-d09a-4af5-85ff-24816d072d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:37.339664 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3701e-d09a-4af5-85ff-24816d072d80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:37.409117 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:37.416364 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2503d8d0-91df-4e69-8a38-de8467e66bae None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:37.417554 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:37.464201 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f3701e-d09a-4af5-85ff-24816d072d80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:37.466167 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f3701e-d09a-4af5-85ff-24816d072d80 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:37.476205 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48b2ac-56f9-4539-a911-3dfce0bb253d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:37.484734 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48b2ac-56f9-4539-a911-3dfce0bb253d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:37.485492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48b2ac-56f9-4539-a911-3dfce0bb253d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:37.485966 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48b2ac-56f9-4539-a911-3dfce0bb253d None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:37.486998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48b2ac-56f9-4539-a911-3dfce0bb253d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:37.487535 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48b2ac-56f9-4539-a911-3dfce0bb253d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:37.489673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48b2ac-56f9-4539-a911-3dfce0bb253d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:37.509678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 133/262] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:37 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:37.516339 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48b2ac-56f9-4539-a911-3dfce0bb253d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:37.656802 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 130/263] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:18:37 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 10:18:37.663334 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f3701e-d09a-4af5-85ff-24816d072d80 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:37.789811 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48b2ac-56f9-4539-a911-3dfce0bb253d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:37.816179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:37.879323 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9373fb5-6b67-42ca-848e-76a98958c1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:37.880128 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9373fb5-6b67-42ca-848e-76a98958c1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:37.880424 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9373fb5-6b67-42ca-848e-76a98958c1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:37.880699 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9373fb5-6b67-42ca-848e-76a98958c1b1 None admin] PATH_INFO: `/v3/users/778d00f8aafb4621be23568946850956` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:37.884644 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9373fb5-6b67-42ca-848e-76a98958c1b1 None admin] RBAC: Authorizing `identity:get_user(user_id=778d00f8aafb4621be23568946850956)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:37.886262 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9373fb5-6b67-42ca-848e-76a98958c1b1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:37.888388 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 134/264] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:37 2022] GET /identity/v3/users/778d00f8aafb4621be23568946850956 => generated 284 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:37.896258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9373fb5-6b67-42ca-848e-76a98958c1b1 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:37.907337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f3701e-d09a-4af5-85ff-24816d072d80 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:37.976022 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf2579-021d-4cb4-be9d-1b7e0d104be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:37.976716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf2579-021d-4cb4-be9d-1b7e0d104be3 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:37.977003 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf2579-021d-4cb4-be9d-1b7e0d104be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:37.977272 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf2579-021d-4cb4-be9d-1b7e0d104be3 None admin] PATH_INFO: `/v3/projects/39e472078e7d42f59243230e66c387f8` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:37.979716 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:37.982348 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf2579-021d-4cb4-be9d-1b7e0d104be3 None admin] RBAC: Authorizing `identity:get_project(project_id=39e472078e7d42f59243230e66c387f8)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:37.984491 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf2579-021d-4cb4-be9d-1b7e0d104be3 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:37.986715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 131/265] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:37 2022] GET /identity/v3/projects/39e472078e7d42f59243230e66c387f8 => generated 303 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:37.994285 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf2579-021d-4cb4-be9d-1b7e0d104be3 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:38.177548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9373fb5-6b67-42ca-848e-76a98958c1b1 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:38.203845 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf2579-021d-4cb4-be9d-1b7e0d104be3 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:38.253097 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eed35c-fe57-48e6-8ce6-746eb61d4566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:38.253756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eed35c-fe57-48e6-8ce6-746eb61d4566 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:38.254044 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eed35c-fe57-48e6-8ce6-746eb61d4566 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:38.254321 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eed35c-fe57-48e6-8ce6-746eb61d4566 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:38.284881 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54df948-6b33-4898-b8ae-4a8b197fb17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:38.286024 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54df948-6b33-4898-b8ae-4a8b197fb17f None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:38.286485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54df948-6b33-4898-b8ae-4a8b197fb17f None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:38.286820 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54df948-6b33-4898-b8ae-4a8b197fb17f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:38.287671 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54df948-6b33-4898-b8ae-4a8b197fb17f None admin] RBAC: Adding query filter params (role.id=89b847fca3574b9083287279aee0f773, scope.project.id=39e472078e7d42f59243230e66c387f8, user.id=778d00f8aafb4621be23568946850956) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:38.288398 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54df948-6b33-4898-b8ae-4a8b197fb17f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:38.290509 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54df948-6b33-4898-b8ae-4a8b197fb17f None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:38.294622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:38.299511 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 132/266] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:18:37 2022] GET /identity/v3/role_assignments?user.id=778d00f8aafb4621be23568946850956&scope.project.id=39e472078e7d42f59243230e66c387f8&role.id=89b847fca3574b9083287279aee0f773 => generated 610 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:18:38.301419 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2463f-8a58-4204-9966-f58d2768f7c6 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:38.309278 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2463f-8a58-4204-9966-f58d2768f7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:38.309278 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2463f-8a58-4204-9966-f58d2768f7c6 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:38.311406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 133/267] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:38 2022] GET /identity => generated 271 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:38.332235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be6487-5c36-4b9d-8668-1dde56d6725b None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:38.332711 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be6487-5c36-4b9d-8668-1dde56d6725b None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:38.333172 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be6487-5c36-4b9d-8668-1dde56d6725b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:38.365094 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-81be6487-5c36-4b9d-8668-1dde56d6725b None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:38.366235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-81be6487-5c36-4b9d-8668-1dde56d6725b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:38.401512 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eed35c-fe57-48e6-8ce6-746eb61d4566 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:38.405128 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eed35c-fe57-48e6-8ce6-746eb61d4566 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:38.479140 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:38.531658 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 134/268] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:38 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:18:38.533223 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb20aa4-c4aa-4e19-ade6-9645d099c2fd None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:38.533515 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb20aa4-c4aa-4e19-ade6-9645d099c2fd None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:38.533788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb20aa4-c4aa-4e19-ade6-9645d099c2fd None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:38.535370 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 135/269] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:38.543011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa46e8-c9cd-400c-b049-9e475ee63ae9 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:38.543283 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa46e8-c9cd-400c-b049-9e475ee63ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:38.543562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa46e8-c9cd-400c-b049-9e475ee63ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:38.583977 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-eefa46e8-c9cd-400c-b049-9e475ee63ae9 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:38.583977 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-eefa46e8-c9cd-400c-b049-9e475ee63ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:38.615372 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 135/270] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:18:37 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 720 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 10:18:38.617131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eed35c-fe57-48e6-8ce6-746eb61d4566 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:38.686040 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 136/271] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:38 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:18:38.687778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa46e8-c9cd-400c-b049-9e475ee63ae9 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:38.775763 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:38.803043 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eed35c-fe57-48e6-8ce6-746eb61d4566 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:38.833773 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa46e8-c9cd-400c-b049-9e475ee63ae9 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:38.842115 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:38.848221 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea0f1e-c5f2-4016-81b5-7e469bd2abc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:38.849019 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea0f1e-c5f2-4016-81b5-7e469bd2abc0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:38.849547 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea0f1e-c5f2-4016-81b5-7e469bd2abc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:38.850023 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea0f1e-c5f2-4016-81b5-7e469bd2abc0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:38.851541 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea0f1e-c5f2-4016-81b5-7e469bd2abc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:38.853075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea0f1e-c5f2-4016-81b5-7e469bd2abc0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:38.856589 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 136/272] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:18:38 2022] GET /identity/v3/domains/default => generated 200 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:38.856589 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea0f1e-c5f2-4016-81b5-7e469bd2abc0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:38.891602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd9eeb-5d80-40ba-82a1-0f3831e58d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:38.892343 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd9eeb-5d80-40ba-82a1-0f3831e58d5b None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:38.892629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd9eeb-5d80-40ba-82a1-0f3831e58d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:38.892909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd9eeb-5d80-40ba-82a1-0f3831e58d5b None admin] PATH_INFO: `/v3/projects/39e472078e7d42f59243230e66c387f8` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:38.894398 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd9eeb-5d80-40ba-82a1-0f3831e58d5b None admin] RBAC: Authorizing `identity:get_project(project_id=39e472078e7d42f59243230e66c387f8)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:38.895761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd9eeb-5d80-40ba-82a1-0f3831e58d5b None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:38.902749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 137/273] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:38 2022] GET /identity/v3/projects/39e472078e7d42f59243230e66c387f8 => generated 303 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:38.903974 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a2376-d98d-4f15-a8a2-6b948dd8c496 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:38.904297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a2376-d98d-4f15-a8a2-6b948dd8c496 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:38.904571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a2376-d98d-4f15-a8a2-6b948dd8c496 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:38.907826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 138/274] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:38.907826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a2376-d98d-4f15-a8a2-6b948dd8c496 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:38.998578 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea0f1e-c5f2-4016-81b5-7e469bd2abc0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:39.042443 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a2376-d98d-4f15-a8a2-6b948dd8c496 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:39.046509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9a84d-5f39-4bdd-a84c-4193d7ab5f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:39.047316 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9a84d-5f39-4bdd-a84c-4193d7ab5f5c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:39.047650 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9a84d-5f39-4bdd-a84c-4193d7ab5f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:39.048036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9a84d-5f39-4bdd-a84c-4193d7ab5f5c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:39.061982 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9a84d-5f39-4bdd-a84c-4193d7ab5f5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:39.067121 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-8cd9a84d-5f39-4bdd-a84c-4193d7ab5f5c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:39.067121 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:39.067121 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:39.067121 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:39.067121 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:39.067121 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:39.067121 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:39.067121 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:39.067121 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:39.067121 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:39.067121 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:39.067121 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:39.067121 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:39.067121 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:39.067121 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:39.067121 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:39.068122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:39.068122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:39.068122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:39.068122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:39.068122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:39.068122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:39.068122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:39.068122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:39.068122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:39.068122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:39.068122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:39.068122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:39.068122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:39.068122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:39.068122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:39.068122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:39.068991 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:39.068991 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:39.068991 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:39.068991 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:39.068991 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:39.068991 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:39.068991 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:39.068991 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:39.068991 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:39.068991 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:39.068991 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:39.068991 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:18:39.068991 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 137/275] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:39.069754 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9a84d-5f39-4bdd-a84c-4193d7ab5f5c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:39.075296 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:39.086598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549b4ac-9fba-41af-b3d3-91e44a3ee34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:39.087279 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549b4ac-9fba-41af-b3d3-91e44a3ee34d None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:39.087563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549b4ac-9fba-41af-b3d3-91e44a3ee34d None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:39.087833 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549b4ac-9fba-41af-b3d3-91e44a3ee34d None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:39.088560 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1549b4ac-9fba-41af-b3d3-91e44a3ee34d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:39.090035 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1549b4ac-9fba-41af-b3d3-91e44a3ee34d None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:39.112657 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 139/276] 10.222.0.121 () {60 vars in 1137 bytes} [Thu Aug 4 10:18:38 2022] POST /identity/v3/users => generated 294 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:18:39.114395 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0080b-2c85-42f3-83f5-19c956ee89b0 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:39.114687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0080b-2c85-42f3-83f5-19c956ee89b0 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:39.114960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0080b-2c85-42f3-83f5-19c956ee89b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:39.136252 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-b5e0080b-2c85-42f3-83f5-19c956ee89b0 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:39.137544 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-b5e0080b-2c85-42f3-83f5-19c956ee89b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:39.163030 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:39.219519 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9a84d-5f39-4bdd-a84c-4193d7ab5f5c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:39.243530 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 140/277] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:39 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:39.245993 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bde79c-0591-4d32-9c60-69bbac0d73ab None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:39.246294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bde79c-0591-4d32-9c60-69bbac0d73ab None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:39.246574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bde79c-0591-4d32-9c60-69bbac0d73ab None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:39.248252 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 141/278] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:39.253389 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bde79c-0591-4d32-9c60-69bbac0d73ab None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:39.279992 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811cc2f5-42c0-4b54-98ec-d2fc1ccc7657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:39.283276 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cc2f5-42c0-4b54-98ec-d2fc1ccc7657 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:39.283276 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cc2f5-42c0-4b54-98ec-d2fc1ccc7657 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:39.283276 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cc2f5-42c0-4b54-98ec-d2fc1ccc7657 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:39.284696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811cc2f5-42c0-4b54-98ec-d2fc1ccc7657 None admin] RBAC: Adding query filter params (role.id=7342106a255b4bce8f150726602a8b95, scope.project.id=39e472078e7d42f59243230e66c387f8, user.id=778d00f8aafb4621be23568946850956) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:39.285218 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811cc2f5-42c0-4b54-98ec-d2fc1ccc7657 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:39.287979 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811cc2f5-42c0-4b54-98ec-d2fc1ccc7657 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:39.301668 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 138/279] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:18:39 2022] GET /identity/v3/role_assignments?user.id=778d00f8aafb4621be23568946850956&scope.project.id=39e472078e7d42f59243230e66c387f8&role.id=7342106a255b4bce8f150726602a8b95 => generated 610 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:39.304182 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0f419-ff5a-4261-a3f6-706f798cc6c4 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:39.304749 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0f419-ff5a-4261-a3f6-706f798cc6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:39.305227 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0f419-ff5a-4261-a3f6-706f798cc6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:39.342046 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-85e0f419-ff5a-4261-a3f6-706f798cc6c4 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:39.343294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-85e0f419-ff5a-4261-a3f6-706f798cc6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:39.435569 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bde79c-0591-4d32-9c60-69bbac0d73ab None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:39.457913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 139/280] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:39 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:39.458616 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0f419-ff5a-4261-a3f6-706f798cc6c4 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:39.462155 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:39.496582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc3772-5f16-4440-9786-d846771b75d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:39.497300 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc3772-5f16-4440-9786-d846771b75d1 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:39.497580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc3772-5f16-4440-9786-d846771b75d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:39.497904 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc3772-5f16-4440-9786-d846771b75d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:39.499783 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc3772-5f16-4440-9786-d846771b75d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:39.499783 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc3772-5f16-4440-9786-d846771b75d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:39.501198 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc3772-5f16-4440-9786-d846771b75d1 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:39.511828 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 142/281] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:39 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:39.512570 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc3772-5f16-4440-9786-d846771b75d1 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:39.548219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:39.631143 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0f419-ff5a-4261-a3f6-706f798cc6c4 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:39.689645 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f186cb0-5c87-4137-833d-d78960e0574b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:39.691406 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f186cb0-5c87-4137-833d-d78960e0574b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:39.691923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f186cb0-5c87-4137-833d-d78960e0574b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:39.693107 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f186cb0-5c87-4137-833d-d78960e0574b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:39.706554 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc3772-5f16-4440-9786-d846771b75d1 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:39.718729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f186cb0-5c87-4137-833d-d78960e0574b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:39.725729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-2f186cb0-5c87-4137-833d-d78960e0574b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:39.725729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:39.725729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:39.725729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:39.725729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:39.725729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:39.725729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:39.725729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:39.725729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:39.725729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:39.725729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:39.725729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:39.725729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:39.725729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:39.725729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:39.725729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:39.726989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:39.726989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:39.726989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:39.726989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:39.726989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:39.726989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:39.726989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:39.726989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:39.726989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:39.726989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:39.726989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:39.726989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:39.726989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:39.726989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:39.726989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:39.726989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:39.727969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:39.727969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:39.727969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:39.727969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:39.727969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:39.727969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:39.727969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:39.727969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:39.727969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:39.727969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:39.727969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:39.727969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:18:39.732737 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 140/282] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:39.733925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f186cb0-5c87-4137-833d-d78960e0574b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:39.775329 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5e74f-d414-4d18-b8e9-658002e9f67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:39.776686 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5e74f-d414-4d18-b8e9-658002e9f67a None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:39.777129 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5e74f-d414-4d18-b8e9-658002e9f67a None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:39.777603 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5e74f-d414-4d18-b8e9-658002e9f67a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:39.794872 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f5e74f-d414-4d18-b8e9-658002e9f67a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:39.804545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-a3f5e74f-d414-4d18-b8e9-658002e9f67a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:18:39.804545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:39.804545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:39.804545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:39.804545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:39.804545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:39.804545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:39.804545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:39.804545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:39.804545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:39.804545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:39.804545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:39.804545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:39.804545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:39.804545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:39.804545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:39.805760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:39.805760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:39.805760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:39.805760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:39.805760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:39.805760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:39.805760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:39.805760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:39.805760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:39.805760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:39.805760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:39.805760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:39.805760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:39.805760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:39.805760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:39.805760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:39.806776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:39.806776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:39.806776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:39.806776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:39.806776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:39.806776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:39.806776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:39.806776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:39.806776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:39.806776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:39.806776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:18:39.806776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:39.810757 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 143/283] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:39.816716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5e74f-d414-4d18-b8e9-658002e9f67a None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:39.862762 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:39.902183 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f186cb0-5c87-4137-833d-d78960e0574b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:39.922499 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:39.951339 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8fe05-b538-43ef-aa85-e29a06c109d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:39.952555 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8fe05-b538-43ef-aa85-e29a06c109d4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:39.953075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8fe05-b538-43ef-aa85-e29a06c109d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:39.953564 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8fe05-b538-43ef-aa85-e29a06c109d4 None admin] PATH_INFO: `/v3/users/ef770655d5114eaf8b9dc20386b99557` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:39.956227 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5e74f-d414-4d18-b8e9-658002e9f67a None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:39.971064 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8fe05-b538-43ef-aa85-e29a06c109d4 None admin] RBAC: Authorizing `identity:get_user(user_id=ef770655d5114eaf8b9dc20386b99557)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:39.973230 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8fe05-b538-43ef-aa85-e29a06c109d4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:39.975714 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 141/284] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:39 2022] GET /identity/v3/users/ef770655d5114eaf8b9dc20386b99557 => generated 294 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:39.977067 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8fe05-b538-43ef-aa85-e29a06c109d4 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:40.007442 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68876d9-04fd-4297-b6d7-5dcf960303fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:40.008706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68876d9-04fd-4297-b6d7-5dcf960303fd None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:40.009201 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68876d9-04fd-4297-b6d7-5dcf960303fd None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:40.009702 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68876d9-04fd-4297-b6d7-5dcf960303fd None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:40.010766 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68876d9-04fd-4297-b6d7-5dcf960303fd None admin] RBAC: Adding query filter params (name=member) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:40.011368 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68876d9-04fd-4297-b6d7-5dcf960303fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:40.013136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68876d9-04fd-4297-b6d7-5dcf960303fd None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:40.024943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 144/285] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:39 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:40.026330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68876d9-04fd-4297-b6d7-5dcf960303fd None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:40.146072 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8fe05-b538-43ef-aa85-e29a06c109d4 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:40.191686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fc1be-218a-4a43-92bf-de5a4c6bbc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:40.192805 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fc1be-218a-4a43-92bf-de5a4c6bbc4f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:40.193315 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fc1be-218a-4a43-92bf-de5a4c6bbc4f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:40.193808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fc1be-218a-4a43-92bf-de5a4c6bbc4f None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:40.194833 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931fc1be-218a-4a43-92bf-de5a4c6bbc4f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:40.195341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931fc1be-218a-4a43-92bf-de5a4c6bbc4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:40.197111 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931fc1be-218a-4a43-92bf-de5a4c6bbc4f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:40.205946 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:40.212245 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 142/286] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:39 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:40.212700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68876d9-04fd-4297-b6d7-5dcf960303fd None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:40.213632 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fc1be-218a-4a43-92bf-de5a4c6bbc4f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:40.252062 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:40.270957 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018315bb-7c40-4ed6-841e-210163680b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:40.271975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018315bb-7c40-4ed6-841e-210163680b56 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:40.272554 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018315bb-7c40-4ed6-841e-210163680b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:40.272935 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018315bb-7c40-4ed6-841e-210163680b56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:40.274280 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018315bb-7c40-4ed6-841e-210163680b56 None admin] RBAC: Adding query filter params (role.id=1b988376d12749bd8d653baa89493c5c, user.id=ef770655d5114eaf8b9dc20386b99557, scope.system=all) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:40.274821 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018315bb-7c40-4ed6-841e-210163680b56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:40.277298 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018315bb-7c40-4ed6-841e-210163680b56 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:40.292351 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 145/287] 10.222.0.121 () {56 vars in 1313 bytes} [Thu Aug 4 10:18:40 2022] GET /identity/v3/role_assignments?user.id=ef770655d5114eaf8b9dc20386b99557&scope.system=all&role.id=1b988376d12749bd8d653baa89493c5c => generated 227 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:40.293937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018315bb-7c40-4ed6-841e-210163680b56 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:40.330043 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fc1be-218a-4a43-92bf-de5a4c6bbc4f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:40.368829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff7330-4c43-4a09-971c-798170d44bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:40.369973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff7330-4c43-4a09-971c-798170d44bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:40.370528 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff7330-4c43-4a09-971c-798170d44bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:40.371005 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff7330-4c43-4a09-971c-798170d44bc2 None admin] PATH_INFO: `/v3/users/6ed2115c84a747d08c732316fa9089dd` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:40.372546 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ff7330-4c43-4a09-971c-798170d44bc2 None admin] RBAC: Authorizing `identity:get_user(user_id=6ed2115c84a747d08c732316fa9089dd)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:40.374425 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ff7330-4c43-4a09-971c-798170d44bc2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:40.376818 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 143/288] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:40 2022] GET /identity/v3/users/6ed2115c84a747d08c732316fa9089dd => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:40.383042 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff7330-4c43-4a09-971c-798170d44bc2 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:40.563911 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff7330-4c43-4a09-971c-798170d44bc2 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:40.567189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:40.575497 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018315bb-7c40-4ed6-841e-210163680b56 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:40.672797 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f35a97-2b05-4e25-85ad-a18570748990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:40.673840 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f35a97-2b05-4e25-85ad-a18570748990 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:40.674315 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f35a97-2b05-4e25-85ad-a18570748990 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:40.674797 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f35a97-2b05-4e25-85ad-a18570748990 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:40.687385 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164afaf9-320c-4450-9271-ef8650ba56ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:40.688611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164afaf9-320c-4450-9271-ef8650ba56ea None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:40.689121 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164afaf9-320c-4450-9271-ef8650ba56ea None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:40.689611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164afaf9-320c-4450-9271-ef8650ba56ea None admin] PATH_INFO: `/v3/users/b21d239600dc4423a7c794c39d189104` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:40.691771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164afaf9-320c-4450-9271-ef8650ba56ea None admin] RBAC: Authorizing `identity:get_user(user_id=b21d239600dc4423a7c794c39d189104)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:40.694598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164afaf9-320c-4450-9271-ef8650ba56ea None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:40.698239 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 146/289] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:40 2022] GET /identity/v3/users/b21d239600dc4423a7c794c39d189104 => generated 298 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:40.705339 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164afaf9-320c-4450-9271-ef8650ba56ea None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:40.752723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:40.836570 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164afaf9-320c-4450-9271-ef8650ba56ea None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:40.877121 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c8c1b-dbe3-4b7c-a365-5cd19699afd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:40.877798 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c8c1b-dbe3-4b7c-a365-5cd19699afd0 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:40.878083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c8c1b-dbe3-4b7c-a365-5cd19699afd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:40.878534 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c8c1b-dbe3-4b7c-a365-5cd19699afd0 None admin] PATH_INFO: `/v3/projects/39e472078e7d42f59243230e66c387f8` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:40.879810 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c8c1b-dbe3-4b7c-a365-5cd19699afd0 None admin] RBAC: Authorizing `identity:get_project(project_id=39e472078e7d42f59243230e66c387f8)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:40.881408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c8c1b-dbe3-4b7c-a365-5cd19699afd0 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:40.883558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 147/290] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:40 2022] GET /identity/v3/projects/39e472078e7d42f59243230e66c387f8 => generated 303 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:40.889329 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c8c1b-dbe3-4b7c-a365-5cd19699afd0 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:40.911351 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f35a97-2b05-4e25-85ad-a18570748990 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:40.915931 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f35a97-2b05-4e25-85ad-a18570748990 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:41.114566 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:41.118533 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 144/291] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:18:40 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 735 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:18:41.123965 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c8c1b-dbe3-4b7c-a365-5cd19699afd0 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:41.127094 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f35a97-2b05-4e25-85ad-a18570748990 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:41.130157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:41.249871 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77e02d-7de6-4248-94f5-94b9f9f1b3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:41.250597 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77e02d-7de6-4248-94f5-94b9f9f1b3de None admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:41.250884 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77e02d-7de6-4248-94f5-94b9f9f1b3de None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:41.251168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77e02d-7de6-4248-94f5-94b9f9f1b3de None admin] PATH_INFO: `/v3/projects/39e472078e7d42f59243230e66c387f8/users/b21d239600dc4423a7c794c39d189104/roles/c77f1c570e8640a3b2c836a401b91889` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:41.257579 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db77e02d-7de6-4248-94f5-94b9f9f1b3de None admin] RBAC: Authorizing `identity:create_grant(project_id=39e472078e7d42f59243230e66c387f8, user_id=b21d239600dc4423a7c794c39d189104, role_id=c77f1c570e8640a3b2c836a401b91889)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:41.259101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db77e02d-7de6-4248-94f5-94b9f9f1b3de None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:41.286774 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 148/292] 10.222.0.121 () {58 vars in 1444 bytes} [Thu Aug 4 10:18:40 2022] PUT /identity/v3/projects/39e472078e7d42f59243230e66c387f8/users/b21d239600dc4423a7c794c39d189104/roles/c77f1c570e8640a3b2c836a401b91889 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:18:41.414915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f35a97-2b05-4e25-85ad-a18570748990 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:41.499113 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79af89-26b9-44c2-b74f-e287a12239c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:41.500363 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79af89-26b9-44c2-b74f-e287a12239c2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:41.500864 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79af89-26b9-44c2-b74f-e287a12239c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:41.501375 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79af89-26b9-44c2-b74f-e287a12239c2 None admin] PATH_INFO: `/v3/projects/39e472078e7d42f59243230e66c387f8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:41.504834 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd79af89-26b9-44c2-b74f-e287a12239c2 None admin] RBAC: Authorizing `identity:get_project(project_id=39e472078e7d42f59243230e66c387f8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:41.506713 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd79af89-26b9-44c2-b74f-e287a12239c2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:41.509179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 145/293] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:41 2022] GET /identity/v3/projects/39e472078e7d42f59243230e66c387f8 => generated 303 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:41.515065 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77e02d-7de6-4248-94f5-94b9f9f1b3de None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:41.659907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77e02d-7de6-4248-94f5-94b9f9f1b3de None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:41.706642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01ab7c-bd39-40d4-9638-638e568c81ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:41.707203 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01ab7c-bd39-40d4-9638-638e568c81ea None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:41.707489 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01ab7c-bd39-40d4-9638-638e568c81ea None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:41.707769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01ab7c-bd39-40d4-9638-638e568c81ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:41.708628 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc01ab7c-bd39-40d4-9638-638e568c81ea None admin] RBAC: Adding query filter params (role.id=1b988376d12749bd8d653baa89493c5c, scope.project.id=39e472078e7d42f59243230e66c387f8, user.id=6ed2115c84a747d08c732316fa9089dd) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:41.708946 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc01ab7c-bd39-40d4-9638-638e568c81ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:41.710507 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc01ab7c-bd39-40d4-9638-638e568c81ea None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:41.717751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 149/294] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:18:41 2022] GET /identity/v3/role_assignments?user.id=6ed2115c84a747d08c732316fa9089dd&scope.project.id=39e472078e7d42f59243230e66c387f8&role.id=1b988376d12749bd8d653baa89493c5c => generated 610 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:41.793907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53a5ec-2037-4503-828e-14651ef0a7a6 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:41.795059 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53a5ec-2037-4503-828e-14651ef0a7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:41.795555 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53a5ec-2037-4503-828e-14651ef0a7a6 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:41.797374 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 146/295] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:41 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:41.804029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3849cb6-a2e7-43d8-b62a-d347ee66cd36 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:41.804672 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3849cb6-a2e7-43d8-b62a-d347ee66cd36 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:41.804672 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3849cb6-a2e7-43d8-b62a-d347ee66cd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:41.833349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-c3849cb6-a2e7-43d8-b62a-d347ee66cd36 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:41.834667 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c3849cb6-a2e7-43d8-b62a-d347ee66cd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:41.855878 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:41.918554 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 150/296] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:41 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:41.943899 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53a5ec-2037-4503-828e-14651ef0a7a6 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:42.029643 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:42.074616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d65e64-0501-47b2-888b-dade35512933 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:42.074900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d65e64-0501-47b2-888b-dade35512933 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:42.075278 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d65e64-0501-47b2-888b-dade35512933 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:42.076966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 151/297] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:42.078747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53a5ec-2037-4503-828e-14651ef0a7a6 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:42.087277 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe8010-4e58-4365-84b8-c6b4d33971ef None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:42.087997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe8010-4e58-4365-84b8-c6b4d33971ef None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:42.088506 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe8010-4e58-4365-84b8-c6b4d33971ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:42.109053 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-2ffe8010-4e58-4365-84b8-c6b4d33971ef None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:42.110731 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-2ffe8010-4e58-4365-84b8-c6b4d33971ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:42.128832 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe5a08-e211-4238-a8d1-5dca12a0382b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:42.130252 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe5a08-e211-4238-a8d1-5dca12a0382b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:42.130548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe5a08-e211-4238-a8d1-5dca12a0382b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:42.130829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe5a08-e211-4238-a8d1-5dca12a0382b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:42.153246 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe5a08-e211-4238-a8d1-5dca12a0382b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:42.159294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-c0fe5a08-e211-4238-a8d1-5dca12a0382b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:18:42.159294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:42.159294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:42.159294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:42.159294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:42.159294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:42.159294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:42.159294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:42.159294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:42.159294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:42.159294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:42.159294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:42.159294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:42.159294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:42.159294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:42.159294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:42.160320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:42.160320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:42.160320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:42.160320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:42.160320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:42.160320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:42.160320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:42.160320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:42.160320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:42.160320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:42.160320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:42.160320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:42.160320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:42.160320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:42.160320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:42.160320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:42.161158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:42.161158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:42.161158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:42.161158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:42.161158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:42.161158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:42.161158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:42.161158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:42.161158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:42.161158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:42.161158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:18:42.161158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:18:42.162499 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 147/298] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:42.170089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe5a08-e211-4238-a8d1-5dca12a0382b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:42.232169 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 152/299] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:42 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:18:42.262441 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe8010-4e58-4365-84b8-c6b4d33971ef None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:42.322756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe5a08-e211-4238-a8d1-5dca12a0382b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:42.348542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:42.358986 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:42.367425 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4454a01-4839-4dd4-a792-83f1dee49c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:42.368508 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4454a01-4839-4dd4-a792-83f1dee49c60 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:42.368950 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4454a01-4839-4dd4-a792-83f1dee49c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:42.369069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe8010-4e58-4365-84b8-c6b4d33971ef None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:42.369480 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4454a01-4839-4dd4-a792-83f1dee49c60 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:42.370393 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4454a01-4839-4dd4-a792-83f1dee49c60 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:42.370911 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4454a01-4839-4dd4-a792-83f1dee49c60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:42.372630 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4454a01-4839-4dd4-a792-83f1dee49c60 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:42.382888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 148/300] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:42 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:42.389023 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4454a01-4839-4dd4-a792-83f1dee49c60 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:42.412063 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb612f91-f264-4e45-8775-fb08a755da30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:42.412786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb612f91-f264-4e45-8775-fb08a755da30 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:42.413079 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb612f91-f264-4e45-8775-fb08a755da30 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:42.413362 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb612f91-f264-4e45-8775-fb08a755da30 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:42.414556 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb612f91-f264-4e45-8775-fb08a755da30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:42.416926 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb612f91-f264-4e45-8775-fb08a755da30 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:42.420106 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 153/301] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:18:42 2022] GET /identity/v3/domains/default => generated 200 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:42.425126 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb612f91-f264-4e45-8775-fb08a755da30 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:42.525961 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4454a01-4839-4dd4-a792-83f1dee49c60 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:42.564454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb612f91-f264-4e45-8775-fb08a755da30 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:42.590192 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c11fa-2ecc-4e9b-98db-7b52a694de3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:42.591451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c11fa-2ecc-4e9b-98db-7b52a694de3c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:42.591971 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c11fa-2ecc-4e9b-98db-7b52a694de3c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:42.592523 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c11fa-2ecc-4e9b-98db-7b52a694de3c None admin] PATH_INFO: `/v3/users/72217dde07a54e9bad663fa3db183281` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:42.621899 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c11fa-2ecc-4e9b-98db-7b52a694de3c None admin] RBAC: Authorizing `identity:get_user(user_id=72217dde07a54e9bad663fa3db183281)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:42.625508 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c11fa-2ecc-4e9b-98db-7b52a694de3c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:42.634170 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 149/302] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:42 2022] GET /identity/v3/users/72217dde07a54e9bad663fa3db183281 => generated 294 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:42.638288 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9648f7-8a9d-4fb5-8019-96e8d8ae8af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:42.639441 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9648f7-8a9d-4fb5-8019-96e8d8ae8af8 None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:42.640007 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9648f7-8a9d-4fb5-8019-96e8d8ae8af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:42.640284 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c11fa-2ecc-4e9b-98db-7b52a694de3c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:42.640861 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9648f7-8a9d-4fb5-8019-96e8d8ae8af8 None admin] PATH_INFO: `/v3/groups` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:42.641890 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9648f7-8a9d-4fb5-8019-96e8d8ae8af8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:42.643707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9648f7-8a9d-4fb5-8019-96e8d8ae8af8 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:42.675759 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 154/303] 10.222.0.121 () {60 vars in 1139 bytes} [Thu Aug 4 10:18:42 2022] POST /identity/v3/groups => generated 230 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:18:42.724667 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81726c-968b-41f2-8663-fd0d53aed39f None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:42.725009 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81726c-968b-41f2-8663-fd0d53aed39f None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:42.725451 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81726c-968b-41f2-8663-fd0d53aed39f None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:42.727255 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 155/304] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:42 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 10:18:42.733398 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc41b6a-f771-4fe5-9684-cb3cc8a59724 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:42.733677 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc41b6a-f771-4fe5-9684-cb3cc8a59724 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:42.733945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc41b6a-f771-4fe5-9684-cb3cc8a59724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:42.747174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:42.758101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-8fc41b6a-f771-4fe5-9684-cb3cc8a59724 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:42.759461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-8fc41b6a-f771-4fe5-9684-cb3cc8a59724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:42.763033 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:42.830192 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c11fa-2ecc-4e9b-98db-7b52a694de3c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:42.884080 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 156/305] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:42 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:18:42.897691 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ebafb2-f34f-4916-905b-ddfea2bb4ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:42.898352 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ebafb2-f34f-4916-905b-ddfea2bb4ceb None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:42.898686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ebafb2-f34f-4916-905b-ddfea2bb4ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:42.898959 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ebafb2-f34f-4916-905b-ddfea2bb4ceb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:42.899999 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ebafb2-f34f-4916-905b-ddfea2bb4ceb None admin] RBAC: Adding query filter params (role.id=c77f1c570e8640a3b2c836a401b91889, user.id=72217dde07a54e9bad663fa3db183281, scope.system=all) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:42.900296 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ebafb2-f34f-4916-905b-ddfea2bb4ceb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:42.901833 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ebafb2-f34f-4916-905b-ddfea2bb4ceb None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:42.905970 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc41b6a-f771-4fe5-9684-cb3cc8a59724 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:42.911503 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 150/306] 10.222.0.121 () {56 vars in 1313 bytes} [Thu Aug 4 10:18:42 2022] GET /identity/v3/role_assignments?user.id=72217dde07a54e9bad663fa3db183281&scope.system=all&role.id=c77f1c570e8640a3b2c836a401b91889 => generated 227 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:43.055504 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc41b6a-f771-4fe5-9684-cb3cc8a59724 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:43.100941 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575e315-4d8f-47d1-843b-abc6faed63b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:43.101970 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575e315-4d8f-47d1-843b-abc6faed63b7 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:43.102444 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575e315-4d8f-47d1-843b-abc6faed63b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:43.102888 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575e315-4d8f-47d1-843b-abc6faed63b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:43.114330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:43.117773 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e575e315-4d8f-47d1-843b-abc6faed63b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:43.123483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-e575e315-4d8f-47d1-843b-abc6faed63b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:43.123483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:43.123483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:43.123483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:43.123483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:43.123483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:43.123483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:43.123483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:43.123483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:43.123483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:43.123483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:43.123483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:43.123483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:43.123483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:43.123483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:43.123483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:43.125130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:43.125130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:43.125130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:43.125130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:43.125130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:43.125130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:43.125130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:43.125130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:43.125130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:43.125130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:43.125130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:43.125130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:43.125130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:43.125130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:43.125130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:43.125130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:43.126588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:43.126588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:43.126588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:43.126588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:43.126588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:43.126588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:43.126588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:43.126588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:43.126588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:43.126588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:43.126588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:43.126588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:43.130409 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 157/307] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:18:43.139045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ebafb2-f34f-4916-905b-ddfea2bb4ceb None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:43.321712 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ebafb2-f34f-4916-905b-ddfea2bb4ceb None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:43.344683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:43.372799 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd01e31-6a37-4435-87e9-e18469259b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:43.373446 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd01e31-6a37-4435-87e9-e18469259b16 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:43.373734 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd01e31-6a37-4435-87e9-e18469259b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:43.374014 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd01e31-6a37-4435-87e9-e18469259b16 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:43.374761 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd01e31-6a37-4435-87e9-e18469259b16 None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:43.375073 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd01e31-6a37-4435-87e9-e18469259b16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:43.376538 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd01e31-6a37-4435-87e9-e18469259b16 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:43.387252 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 151/308] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:43 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:43.396226 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575e315-4d8f-47d1-843b-abc6faed63b7 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:43.520215 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575e315-4d8f-47d1-843b-abc6faed63b7 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:43.565384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba018075-2160-4f9b-9a8f-39c01b3299eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:43.566080 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba018075-2160-4f9b-9a8f-39c01b3299eb None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:43.566370 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba018075-2160-4f9b-9a8f-39c01b3299eb None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:43.566683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba018075-2160-4f9b-9a8f-39c01b3299eb None admin] PATH_INFO: `/v3/users/ef770655d5114eaf8b9dc20386b99557` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:43.568081 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba018075-2160-4f9b-9a8f-39c01b3299eb None admin] RBAC: Authorizing `identity:get_user(user_id=ef770655d5114eaf8b9dc20386b99557)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:43.570039 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba018075-2160-4f9b-9a8f-39c01b3299eb None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:43.572171 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 158/309] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:43 2022] GET /identity/v3/users/ef770655d5114eaf8b9dc20386b99557 => generated 294 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:43.585396 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd01e31-6a37-4435-87e9-e18469259b16 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:43.699909 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd01e31-6a37-4435-87e9-e18469259b16 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:43.737711 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5660366-437f-425d-89fb-a96eb3801ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:43.738776 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5660366-437f-425d-89fb-a96eb3801ccf None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:43.739066 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5660366-437f-425d-89fb-a96eb3801ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:43.739343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5660366-437f-425d-89fb-a96eb3801ccf None admin] PATH_INFO: `/v3/system/users/ef770655d5114eaf8b9dc20386b99557/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:43.740982 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5660366-437f-425d-89fb-a96eb3801ccf None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ef770655d5114eaf8b9dc20386b99557, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:43.742415 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5660366-437f-425d-89fb-a96eb3801ccf None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:43.754242 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 152/310] 10.222.0.121 () {58 vars in 1339 bytes} [Thu Aug 4 10:18:43 2022] PUT /identity/v3/system/users/ef770655d5114eaf8b9dc20386b99557/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:18:44.165347 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25121c48-7acf-4f21-a80a-0fa12c5f5ff9 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:44.165981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25121c48-7acf-4f21-a80a-0fa12c5f5ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:44.165981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25121c48-7acf-4f21-a80a-0fa12c5f5ff9 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:44.167578 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 159/311] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:44.173651 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00c722-645b-49dd-9cfa-729e1072f387 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:44.174242 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00c722-645b-49dd-9cfa-729e1072f387 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:44.174242 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00c722-645b-49dd-9cfa-729e1072f387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:44.195337 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-da00c722-645b-49dd-9cfa-729e1072f387 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:44.196621 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-da00c722-645b-49dd-9cfa-729e1072f387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:44.236235 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:44.273549 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 153/312] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:44 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:44.296886 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25121c48-7acf-4f21-a80a-0fa12c5f5ff9 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:44.374740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:44.412651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25121c48-7acf-4f21-a80a-0fa12c5f5ff9 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:44.454284 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474379f2-7002-4965-8e58-94b6f53632f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:44.454857 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474379f2-7002-4965-8e58-94b6f53632f7 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:44.455143 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474379f2-7002-4965-8e58-94b6f53632f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:44.455424 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474379f2-7002-4965-8e58-94b6f53632f7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:44.466523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474379f2-7002-4965-8e58-94b6f53632f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:44.471629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-474379f2-7002-4965-8e58-94b6f53632f7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:18:44.471629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:44.471629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:44.471629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:44.471629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:44.471629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:44.471629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:44.471629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:44.471629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:44.471629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:44.471629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:44.471629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:44.471629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:44.471629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:44.471629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:44.471629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:44.473907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:44.473907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:44.473907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:44.473907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:44.473907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:44.473907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:44.473907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:44.473907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:44.473907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:44.473907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:44.473907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:44.473907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:44.473907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:44.473907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:44.473907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:44.473907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:44.476117 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:44.476117 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:44.476117 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:44.476117 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:44.476117 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:44.476117 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:44.476117 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:44.476117 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:44.476117 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:44.476117 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:44.476117 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:18:44.476117 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:44.476117 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 160/313] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:44.481454 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00c722-645b-49dd-9cfa-729e1072f387 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:44.595488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00c722-645b-49dd-9cfa-729e1072f387 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:44.634080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ce27b-e1ff-44ad-9c60-68968987f457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:44.634736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ce27b-e1ff-44ad-9c60-68968987f457 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:44.635023 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ce27b-e1ff-44ad-9c60-68968987f457 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:44.635304 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ce27b-e1ff-44ad-9c60-68968987f457 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:44.636194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ce27b-e1ff-44ad-9c60-68968987f457 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:44.636508 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ce27b-e1ff-44ad-9c60-68968987f457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:44.637960 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ce27b-e1ff-44ad-9c60-68968987f457 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:44.647797 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 154/314] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:44 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:44.653801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474379f2-7002-4965-8e58-94b6f53632f7 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:44.766506 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474379f2-7002-4965-8e58-94b6f53632f7 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:44.796811 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:44.805942 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61ea94-ccb4-4691-a8ab-fccff94a8a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:44.806649 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61ea94-ccb4-4691-a8ab-fccff94a8a56 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:44.807157 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61ea94-ccb4-4691-a8ab-fccff94a8a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:44.807431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61ea94-ccb4-4691-a8ab-fccff94a8a56 None admin] PATH_INFO: `/v3/users/b21d239600dc4423a7c794c39d189104` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:44.808707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61ea94-ccb4-4691-a8ab-fccff94a8a56 None admin] RBAC: Authorizing `identity:get_user(user_id=b21d239600dc4423a7c794c39d189104)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:44.810219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61ea94-ccb4-4691-a8ab-fccff94a8a56 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:44.811937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 161/315] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:44 2022] GET /identity/v3/users/b21d239600dc4423a7c794c39d189104 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:44.818022 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ce27b-e1ff-44ad-9c60-68968987f457 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:44.855632 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:44.933039 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c30c9-9505-4987-a79c-5dcf0d68c2d5 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:44.934023 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c30c9-9505-4987-a79c-5dcf0d68c2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:44.934388 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c30c9-9505-4987-a79c-5dcf0d68c2d5 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:44.937941 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 162/316] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:44 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:44.943519 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb0237-fe45-485c-ac4c-14429fcd6610 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:44.943847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb0237-fe45-485c-ac4c-14429fcd6610 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:44.944444 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb0237-fe45-485c-ac4c-14429fcd6610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:44.959369 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ce27b-e1ff-44ad-9c60-68968987f457 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:44.970444 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-d0eb0237-fe45-485c-ac4c-14429fcd6610 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:44.971979 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d0eb0237-fe45-485c-ac4c-14429fcd6610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:45.010722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb294550-471d-445d-8978-228181df8a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:45.012203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb294550-471d-445d-8978-228181df8a1e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:45.012508 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb294550-471d-445d-8978-228181df8a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:45.012792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb294550-471d-445d-8978-228181df8a1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:45.076418 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 163/317] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:44 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:18:45.090941 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb0237-fe45-485c-ac4c-14429fcd6610 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:45.121340 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb294550-471d-445d-8978-228181df8a1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:45.123646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb294550-471d-445d-8978-228181df8a1e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:45.166958 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:45.216935 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb0237-fe45-485c-ac4c-14429fcd6610 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:45.251345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:45.261167 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550fc11-5a08-4945-8d06-70c49a6bc178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:45.262095 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550fc11-5a08-4945-8d06-70c49a6bc178 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:45.262508 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550fc11-5a08-4945-8d06-70c49a6bc178 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:45.262879 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550fc11-5a08-4945-8d06-70c49a6bc178 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:45.264322 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550fc11-5a08-4945-8d06-70c49a6bc178 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:45.265948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550fc11-5a08-4945-8d06-70c49a6bc178 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:45.273055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 164/318] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:18:45 2022] GET /identity/v3/domains/default => generated 200 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:18:45.274034 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550fc11-5a08-4945-8d06-70c49a6bc178 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:45.294903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 155/319] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:18:44 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:18:45.303157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb294550-471d-445d-8978-228181df8a1e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:45.405678 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550fc11-5a08-4945-8d06-70c49a6bc178 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:45.447575 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a7203-f663-49e3-8e29-70d170e9c5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:45.448658 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a7203-f663-49e3-8e29-70d170e9c5e4 None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:45.449103 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a7203-f663-49e3-8e29-70d170e9c5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:45.449494 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a7203-f663-49e3-8e29-70d170e9c5e4 None admin] PATH_INFO: `/v3/groups` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:45.450647 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a7203-f663-49e3-8e29-70d170e9c5e4 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:45.453290 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a7203-f663-49e3-8e29-70d170e9c5e4 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:45.454456 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb294550-471d-445d-8978-228181df8a1e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:45.484193 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 165/320] 10.222.0.121 () {60 vars in 1139 bytes} [Thu Aug 4 10:18:45 2022] POST /identity/v3/groups => generated 227 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:18:45.486687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c7a55-d328-45ef-b329-7fc962a3bc55 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:45.487231 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c7a55-d328-45ef-b329-7fc962a3bc55 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:45.488651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c7a55-d328-45ef-b329-7fc962a3bc55 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:45.496560 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 166/321] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:45 2022] GET /identity => generated 271 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:45.500511 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce881dd0-a0e5-4520-bd63-0035b46954f4 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:45.500926 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce881dd0-a0e5-4520-bd63-0035b46954f4 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:45.501340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce881dd0-a0e5-4520-bd63-0035b46954f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:45.513112 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:45.519975 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d659f-babb-4d12-9648-2361158985f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:45.521068 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d659f-babb-4d12-9648-2361158985f8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:45.521526 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d659f-babb-4d12-9648-2361158985f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:45.521812 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d659f-babb-4d12-9648-2361158985f8 None admin] PATH_INFO: `/v3/projects/39e472078e7d42f59243230e66c387f8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:45.523219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d659f-babb-4d12-9648-2361158985f8 None admin] RBAC: Authorizing `identity:get_project(project_id=39e472078e7d42f59243230e66c387f8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:45.524940 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d659f-babb-4d12-9648-2361158985f8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:45.527500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 156/322] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:45 2022] GET /identity/v3/projects/39e472078e7d42f59243230e66c387f8 => generated 303 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:45.534978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d659f-babb-4d12-9648-2361158985f8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:45.542194 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-ce881dd0-a0e5-4520-bd63-0035b46954f4 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:45.544678 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-ce881dd0-a0e5-4520-bd63-0035b46954f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:45.645357 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:45.676645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 167/323] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:45 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:45.680666 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d659f-babb-4d12-9648-2361158985f8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:45.693375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce881dd0-a0e5-4520-bd63-0035b46954f4 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:45.726860 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288811ce-e720-4775-ba94-d6c87d8ccfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:45.728193 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288811ce-e720-4775-ba94-d6c87d8ccfdf None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:45.728804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288811ce-e720-4775-ba94-d6c87d8ccfdf None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:45.729275 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288811ce-e720-4775-ba94-d6c87d8ccfdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:45.730275 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288811ce-e720-4775-ba94-d6c87d8ccfdf None admin] RBAC: Adding query filter params (role.id=c77f1c570e8640a3b2c836a401b91889, scope.project.id=39e472078e7d42f59243230e66c387f8, user.id=b21d239600dc4423a7c794c39d189104) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:45.730914 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288811ce-e720-4775-ba94-d6c87d8ccfdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:45.732463 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288811ce-e720-4775-ba94-d6c87d8ccfdf None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:45.741652 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 157/324] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:18:45 2022] GET /identity/v3/role_assignments?user.id=b21d239600dc4423a7c794c39d189104&scope.project.id=39e472078e7d42f59243230e66c387f8&role.id=c77f1c570e8640a3b2c836a401b91889 => generated 610 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:18:45.853201 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce881dd0-a0e5-4520-bd63-0035b46954f4 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:45.902629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e933459-4443-4083-bbff-6351f1e62273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:45.903382 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e933459-4443-4083-bbff-6351f1e62273 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:45.903666 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e933459-4443-4083-bbff-6351f1e62273 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:45.904054 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e933459-4443-4083-bbff-6351f1e62273 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:45.922238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e933459-4443-4083-bbff-6351f1e62273 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:45.927538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-5e933459-4443-4083-bbff-6351f1e62273 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:18:45.927538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:45.927538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:45.927538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:45.927538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:45.927538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:45.927538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:45.927538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:45.927538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:45.927538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:45.927538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:45.927538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:45.927538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:45.927538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:45.927538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:45.927538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:45.928574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:45.928574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:45.928574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:45.928574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:45.928574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:45.928574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:45.928574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:45.928574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:45.928574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:45.928574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:45.928574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:45.928574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:45.928574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:45.928574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:45.928574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:45.928574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:45.929689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:45.929689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:45.929689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:45.929689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:45.929689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:45.929689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:45.929689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:45.929689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:45.929689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:45.929689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:45.929689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:18:45.929689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:45.929689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 168/325] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:18:45.935624 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288811ce-e720-4775-ba94-d6c87d8ccfdf None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:45.937295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055d862-e34f-4bad-a524-732d6a68ae01 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:45.937639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055d862-e34f-4bad-a524-732d6a68ae01 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:45.937925 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055d862-e34f-4bad-a524-732d6a68ae01 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:45.940233 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 169/326] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:45 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 10:18:45.945945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00d16c-3fa7-47a1-a6b8-ad3524e804ab None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:45.946255 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00d16c-3fa7-47a1-a6b8-ad3524e804ab None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:45.946511 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00d16c-3fa7-47a1-a6b8-ad3524e804ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:45.967332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-0c00d16c-3fa7-47a1-a6b8-ad3524e804ab None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:45.969523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-0c00d16c-3fa7-47a1-a6b8-ad3524e804ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:46.029061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:46.035327 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:46.058554 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 170/327] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:45 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:18:46.084903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288811ce-e720-4775-ba94-d6c87d8ccfdf None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:46.088644 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00d16c-3fa7-47a1-a6b8-ad3524e804ab None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:46.138794 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f02830-9031-42b7-ad96-68099adcab36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:46.139471 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f02830-9031-42b7-ad96-68099adcab36 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:46.139748 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f02830-9031-42b7-ad96-68099adcab36 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:46.140178 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f02830-9031-42b7-ad96-68099adcab36 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:46.140931 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f02830-9031-42b7-ad96-68099adcab36 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:46.141251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f02830-9031-42b7-ad96-68099adcab36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:46.142700 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f02830-9031-42b7-ad96-68099adcab36 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:46.152894 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 158/328] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:45 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:46.158994 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f02830-9031-42b7-ad96-68099adcab36 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:46.254438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00d16c-3fa7-47a1-a6b8-ad3524e804ab None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:46.308482 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbfa3ec-d310-49e9-b71e-6e9084a4b650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:46.309175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfa3ec-d310-49e9-b71e-6e9084a4b650 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:46.309458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfa3ec-d310-49e9-b71e-6e9084a4b650 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:46.309755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfa3ec-d310-49e9-b71e-6e9084a4b650 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:46.326949 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f02830-9031-42b7-ad96-68099adcab36 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:46.340143 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbfa3ec-d310-49e9-b71e-6e9084a4b650 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:46.350109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-4cbfa3ec-d310-49e9-b71e-6e9084a4b650 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:46.350109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:46.350109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:46.350109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:46.350109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:46.350109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:46.350109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:46.350109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:46.350109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:46.350109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:46.350109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:46.350109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:46.350109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:46.350109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:46.350109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:46.350109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:46.352119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:46.352119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:46.352119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:46.352119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:46.352119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:46.352119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:46.352119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:46.352119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:46.352119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:46.352119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:46.352119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:46.352119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:46.352119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:46.352119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:46.352119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:46.352119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:46.353643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:46.353643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:46.353643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:46.353643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:46.353643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:46.353643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:46.353643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:46.353643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:46.353643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:46.353643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:46.353643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:46.353643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:46.353643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 171/329] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:18:46.361114 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbfa3ec-d310-49e9-b71e-6e9084a4b650 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:46.376666 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:46.392861 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0bf80-1a96-4f71-ba8d-5a97f0eca7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:46.393362 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0bf80-1a96-4f71-ba8d-5a97f0eca7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:46.393651 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0bf80-1a96-4f71-ba8d-5a97f0eca7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:46.393927 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0bf80-1a96-4f71-ba8d-5a97f0eca7f1 None admin] PATH_INFO: `/v3/users/72217dde07a54e9bad663fa3db183281` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:46.398166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0bf80-1a96-4f71-ba8d-5a97f0eca7f1 None admin] RBAC: Authorizing `identity:get_user(user_id=72217dde07a54e9bad663fa3db183281)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:46.399643 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0bf80-1a96-4f71-ba8d-5a97f0eca7f1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:46.404252 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 159/330] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:46 2022] GET /identity/v3/users/72217dde07a54e9bad663fa3db183281 => generated 294 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:18:46.413145 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0bf80-1a96-4f71-ba8d-5a97f0eca7f1 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:46.434396 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:46.541727 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbfa3ec-d310-49e9-b71e-6e9084a4b650 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:46.594644 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659da0ec-9b27-49d2-a3d3-7b9c6e5a1441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:46.595687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659da0ec-9b27-49d2-a3d3-7b9c6e5a1441 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:46.596239 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659da0ec-9b27-49d2-a3d3-7b9c6e5a1441 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:46.596755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659da0ec-9b27-49d2-a3d3-7b9c6e5a1441 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:46.598542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659da0ec-9b27-49d2-a3d3-7b9c6e5a1441 None admin] RBAC: Adding query filter params (name=member) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:46.599127 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659da0ec-9b27-49d2-a3d3-7b9c6e5a1441 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:46.601937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659da0ec-9b27-49d2-a3d3-7b9c6e5a1441 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:46.614251 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 172/331] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:46 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:46.619886 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659da0ec-9b27-49d2-a3d3-7b9c6e5a1441 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:46.673707 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0bf80-1a96-4f71-ba8d-5a97f0eca7f1 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:46.758908 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c020a-1e1e-4298-bd5f-c30cd0c68465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:46.761679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c020a-1e1e-4298-bd5f-c30cd0c68465 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:46.762605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c020a-1e1e-4298-bd5f-c30cd0c68465 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:46.763281 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c020a-1e1e-4298-bd5f-c30cd0c68465 None admin] PATH_INFO: `/v3/system/users/72217dde07a54e9bad663fa3db183281/roles/c77f1c570e8640a3b2c836a401b91889` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:46.770425 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29c020a-1e1e-4298-bd5f-c30cd0c68465 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=72217dde07a54e9bad663fa3db183281, role_id=c77f1c570e8640a3b2c836a401b91889)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:46.774502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29c020a-1e1e-4298-bd5f-c30cd0c68465 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:46.791013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 160/332] 10.222.0.121 () {58 vars in 1339 bytes} [Thu Aug 4 10:18:46 2022] PUT /identity/v3/system/users/72217dde07a54e9bad663fa3db183281/roles/c77f1c570e8640a3b2c836a401b91889 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:18:46.818848 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:46.829334 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659da0ec-9b27-49d2-a3d3-7b9c6e5a1441 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:46.892351 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8384b8d-714a-477b-869b-d573005f6bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:46.893335 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8384b8d-714a-477b-869b-d573005f6bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:46.893707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8384b8d-714a-477b-869b-d573005f6bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:46.894061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8384b8d-714a-477b-869b-d573005f6bc3 None admin] PATH_INFO: `/v3/users/ef770655d5114eaf8b9dc20386b99557` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:46.896167 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8384b8d-714a-477b-869b-d573005f6bc3 None admin] RBAC: Authorizing `identity:get_user(user_id=ef770655d5114eaf8b9dc20386b99557)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:46.898580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8384b8d-714a-477b-869b-d573005f6bc3 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:46.901245 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 173/333] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:46 2022] GET /identity/v3/users/ef770655d5114eaf8b9dc20386b99557 => generated 294 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:18:46.907617 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c020a-1e1e-4298-bd5f-c30cd0c68465 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:47.003384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:47.054307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c020a-1e1e-4298-bd5f-c30cd0c68465 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:47.102329 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08583e-4d6a-48ab-bf8e-a3e14036d686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:47.102932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08583e-4d6a-48ab-bf8e-a3e14036d686 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:47.103350 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08583e-4d6a-48ab-bf8e-a3e14036d686 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:47.103544 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08583e-4d6a-48ab-bf8e-a3e14036d686 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:47.104376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08583e-4d6a-48ab-bf8e-a3e14036d686 None admin] RBAC: Adding query filter params (role.id=1b988376d12749bd8d653baa89493c5c, user.id=ef770655d5114eaf8b9dc20386b99557, scope.system=all) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:47.104696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08583e-4d6a-48ab-bf8e-a3e14036d686 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:47.106221 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08583e-4d6a-48ab-bf8e-a3e14036d686 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:47.122929 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 161/334] 10.222.0.121 () {56 vars in 1313 bytes} [Thu Aug 4 10:18:46 2022] GET /identity/v3/role_assignments?user.id=ef770655d5114eaf8b9dc20386b99557&scope.system=all&role.id=1b988376d12749bd8d653baa89493c5c => generated 512 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:48.624478 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d100f2-a96c-4344-b73c-9679fab0c7de None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:48.625261 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d100f2-a96c-4344-b73c-9679fab0c7de None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:48.625261 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d100f2-a96c-4344-b73c-9679fab0c7de None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:48.626717 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 174/335] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:48 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:48.634231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754a936-a8f0-4950-8508-1b58a5c8bacd None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:48.634941 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754a936-a8f0-4950-8508-1b58a5c8bacd None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:48.634941 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754a936-a8f0-4950-8508-1b58a5c8bacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:48.657046 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a754a936-a8f0-4950-8508-1b58a5c8bacd None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:48.658281 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a754a936-a8f0-4950-8508-1b58a5c8bacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:48.752845 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 162/336] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:48 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:18:48.812825 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d100f2-a96c-4344-b73c-9679fab0c7de None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:48.946037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d100f2-a96c-4344-b73c-9679fab0c7de None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:48.992714 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96775886-4334-43a0-87ca-93b9440e96cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:48.993398 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96775886-4334-43a0-87ca-93b9440e96cd None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:48.993685 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96775886-4334-43a0-87ca-93b9440e96cd None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:48.993963 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96775886-4334-43a0-87ca-93b9440e96cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:49.020960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96775886-4334-43a0-87ca-93b9440e96cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:49.026964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-96775886-4334-43a0-87ca-93b9440e96cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:49.026964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:49.026964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:49.026964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:49.026964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:49.026964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:49.026964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:49.026964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:49.026964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:49.026964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:49.026964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:49.026964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:49.026964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:49.026964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:49.026964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:49.026964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:49.029328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:49.029328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:49.029328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:49.029328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:49.029328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:49.029328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:49.029328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:49.029328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:49.029328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:49.029328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:49.029328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:49.029328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:49.029328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:49.029328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:49.029328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:49.029328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:49.031341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:49.031341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:49.031341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:49.031341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:49.031341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:49.031341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:49.031341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:49.031341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:49.031341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:49.031341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:49.031341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:49.031341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:49.031341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 175/337] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:49.039971 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754a936-a8f0-4950-8508-1b58a5c8bacd None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:49.118939 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:49.192185 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754a936-a8f0-4950-8508-1b58a5c8bacd None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:49.241952 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99acf1b-40a2-4dda-a208-5466fd0eea5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:49.242527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99acf1b-40a2-4dda-a208-5466fd0eea5a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:49.242814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99acf1b-40a2-4dda-a208-5466fd0eea5a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:49.243091 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99acf1b-40a2-4dda-a208-5466fd0eea5a None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:49.243842 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99acf1b-40a2-4dda-a208-5466fd0eea5a None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:49.244176 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99acf1b-40a2-4dda-a208-5466fd0eea5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:49.245615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99acf1b-40a2-4dda-a208-5466fd0eea5a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:49.259922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 163/338] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:49 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:49.263353 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96775886-4334-43a0-87ca-93b9440e96cd None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:49.308081 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:49.405617 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96775886-4334-43a0-87ca-93b9440e96cd None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:49.447638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca9c3e-48c1-42e0-969e-e33f04828136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:49.448736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca9c3e-48c1-42e0-969e-e33f04828136 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:49.449266 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca9c3e-48c1-42e0-969e-e33f04828136 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:49.449755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca9c3e-48c1-42e0-969e-e33f04828136 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:49.456162 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa467a-426b-4bd3-8d18-40e7c346079e None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:49.456481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa467a-426b-4bd3-8d18-40e7c346079e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:49.456731 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa467a-426b-4bd3-8d18-40e7c346079e None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:49.459337 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 164/339] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:49.464235 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da318955-691b-414e-afb7-2317468e3a3b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:49.464522 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da318955-691b-414e-afb7-2317468e3a3b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:49.464805 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da318955-691b-414e-afb7-2317468e3a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:49.508117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-da318955-691b-414e-afb7-2317468e3a3b None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:49.512905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-da318955-691b-414e-afb7-2317468e3a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:49.557304 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca9c3e-48c1-42e0-969e-e33f04828136 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:49.559256 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca9c3e-48c1-42e0-969e-e33f04828136 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:49.588548 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:49.677507 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 176/340] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:18:49 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:18:49.679858 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163f504-9235-4613-85ff-371484255bf6 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:49.680455 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163f504-9235-4613-85ff-371484255bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:49.680960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163f504-9235-4613-85ff-371484255bf6 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:49.682517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 177/341] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:49 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:49.689170 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163f504-9235-4613-85ff-371484255bf6 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:49.700207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 165/342] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:49 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:18:49.701596 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af8bcf-f819-4c9a-b99b-a95ec2b99e2e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:49.701889 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af8bcf-f819-4c9a-b99b-a95ec2b99e2e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:49.702167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af8bcf-f819-4c9a-b99b-a95ec2b99e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:49.735147 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-47af8bcf-f819-4c9a-b99b-a95ec2b99e2e None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:49.736558 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-47af8bcf-f819-4c9a-b99b-a95ec2b99e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:49.789120 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:49.860982 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 166/343] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:49 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:49.861897 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af8bcf-f819-4c9a-b99b-a95ec2b99e2e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:49.882653 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163f504-9235-4613-85ff-371484255bf6 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:49.924349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:49.948645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0896db1-236b-4339-bf4f-3a4c4aacaa68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:49.949322 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0896db1-236b-4339-bf4f-3a4c4aacaa68 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:49.949605 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0896db1-236b-4339-bf4f-3a4c4aacaa68 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:49.949884 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0896db1-236b-4339-bf4f-3a4c4aacaa68 None admin] PATH_INFO: `/v3/projects/6d7a7a0b84384f14849056212228badb` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:49.952012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0896db1-236b-4339-bf4f-3a4c4aacaa68 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7a7a0b84384f14849056212228badb)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:49.954358 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0896db1-236b-4339-bf4f-3a4c4aacaa68 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:49.958099 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 178/344] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:49 2022] GET /identity/v3/projects/6d7a7a0b84384f14849056212228badb => generated 299 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:49.958874 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0896db1-236b-4339-bf4f-3a4c4aacaa68 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:50.058239 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af8bcf-f819-4c9a-b99b-a95ec2b99e2e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:50.103691 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650d4f4-be80-423b-950e-44e09086e5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:50.104740 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650d4f4-be80-423b-950e-44e09086e5b6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:50.105180 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650d4f4-be80-423b-950e-44e09086e5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:50.105605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650d4f4-be80-423b-950e-44e09086e5b6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:50.119217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650d4f4-be80-423b-950e-44e09086e5b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:50.124844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-8650d4f4-be80-423b-950e-44e09086e5b6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:18:50.124844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:50.124844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:50.124844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:50.124844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:50.124844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:50.124844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:50.124844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:50.124844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:50.124844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:50.124844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:50.124844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:50.124844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:50.124844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:50.124844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:50.124844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:50.125859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:50.125859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:50.125859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:50.125859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:50.125859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:50.125859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:50.125859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:50.125859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:50.125859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:50.125859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:50.125859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:50.125859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:50.125859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:50.125859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:50.125859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:50.125859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:50.126712 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:50.126712 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:50.126712 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:50.126712 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:50.126712 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:50.126712 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:50.126712 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:50.126712 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:50.126712 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:50.126712 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:50.126712 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:18:50.126712 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:18:50.129465 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 167/345] 10.222.0.121 () {56 vars in 1119 bytes} [Thu Aug 4 10:18:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:50.130694 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650d4f4-be80-423b-950e-44e09086e5b6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:50.140909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0896db1-236b-4339-bf4f-3a4c4aacaa68 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:50.174752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:50.189569 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca29c8f-15fa-46b1-bc46-01281975dd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:50.190268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca29c8f-15fa-46b1-bc46-01281975dd3f None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:50.190549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca29c8f-15fa-46b1-bc46-01281975dd3f None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:50.190823 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca29c8f-15fa-46b1-bc46-01281975dd3f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:50.208266 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca29c8f-15fa-46b1-bc46-01281975dd3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:50.215567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-2ca29c8f-15fa-46b1-bc46-01281975dd3f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:50.215567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:50.215567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:50.215567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:50.215567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:50.215567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:50.215567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:50.215567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:50.215567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:50.215567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:50.215567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:50.215567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:50.215567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:50.215567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:50.215567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:50.215567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:50.216620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:50.216620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:50.216620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:50.216620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:50.216620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:50.216620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:50.216620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:50.216620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:50.216620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:50.216620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:50.216620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:50.216620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:50.216620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:50.216620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:50.216620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:50.216620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:50.217558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:50.217558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:50.217558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:50.217558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:50.217558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:50.217558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:50.217558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:50.217558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:50.217558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:50.217558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:50.217558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:50.217558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:50.217558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 179/346] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:50.224024 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135681d-54c0-4353-82c4-134a858d3760 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:50.224024 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135681d-54c0-4353-82c4-134a858d3760 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:50.224024 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135681d-54c0-4353-82c4-134a858d3760 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:50.224024 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 180/347] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:50 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:50.224024 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135681d-54c0-4353-82c4-134a858d3760 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:50.290389 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650d4f4-be80-423b-950e-44e09086e5b6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:50.307648 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:50.340449 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd034890-ee3a-4e54-910f-253fadab1ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:50.341130 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd034890-ee3a-4e54-910f-253fadab1ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:50.341863 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd034890-ee3a-4e54-910f-253fadab1ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:50.341863 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd034890-ee3a-4e54-910f-253fadab1ec9 None admin] PATH_INFO: `/v3/groups/6d7d63731f51414185728735a19d856f` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:50.349551 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd034890-ee3a-4e54-910f-253fadab1ec9 None admin] RBAC: Authorizing `identity:get_group(group_id=6d7d63731f51414185728735a19d856f)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:50.351116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd034890-ee3a-4e54-910f-253fadab1ec9 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:50.353001 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 168/348] 10.222.0.121 () {56 vars in 1185 bytes} [Thu Aug 4 10:18:50 2022] GET /identity/v3/groups/6d7d63731f51414185728735a19d856f => generated 227 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:50.353811 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd034890-ee3a-4e54-910f-253fadab1ec9 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:50.389952 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135681d-54c0-4353-82c4-134a858d3760 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:50.440186 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97d310-7d86-465f-a71e-32579d8d3fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:50.440855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97d310-7d86-465f-a71e-32579d8d3fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:50.441137 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97d310-7d86-465f-a71e-32579d8d3fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:50.441414 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97d310-7d86-465f-a71e-32579d8d3fb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:50.442351 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97d310-7d86-465f-a71e-32579d8d3fb4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:50.442664 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97d310-7d86-465f-a71e-32579d8d3fb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:50.446475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97d310-7d86-465f-a71e-32579d8d3fb4 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:50.457735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 181/349] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:18:50 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:50.459601 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7720c6a-62ce-4a80-8c8a-52d6aaecc1e8 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:50.460186 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7720c6a-62ce-4a80-8c8a-52d6aaecc1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:50.460722 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7720c6a-62ce-4a80-8c8a-52d6aaecc1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:50.478506 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:50.483026 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-d7720c6a-62ce-4a80-8c8a-52d6aaecc1e8 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:50.484605 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d7720c6a-62ce-4a80-8c8a-52d6aaecc1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:50.491024 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd034890-ee3a-4e54-910f-253fadab1ec9 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:50.537179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724229f-68bd-415e-9822-868d853f086b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:50.537440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724229f-68bd-415e-9822-868d853f086b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:50.537731 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724229f-68bd-415e-9822-868d853f086b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:50.539053 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724229f-68bd-415e-9822-868d853f086b None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:50.539053 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6724229f-68bd-415e-9822-868d853f086b None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:50.539053 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6724229f-68bd-415e-9822-868d853f086b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:50.541046 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6724229f-68bd-415e-9822-868d853f086b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:50.555693 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 169/350] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:50 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:50.557546 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407c226-7d00-4fd7-b44e-adf3cf291ea6 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:50.558051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407c226-7d00-4fd7-b44e-adf3cf291ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:50.558556 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407c226-7d00-4fd7-b44e-adf3cf291ea6 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:50.560621 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 170/351] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:50 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:50.562211 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4407c226-7d00-4fd7-b44e-adf3cf291ea6 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:50.584970 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 182/352] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:50 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:50.585754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7720c6a-62ce-4a80-8c8a-52d6aaecc1e8 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:50.638103 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:50.709249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4407c226-7d00-4fd7-b44e-adf3cf291ea6 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:50.714244 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7720c6a-62ce-4a80-8c8a-52d6aaecc1e8 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:50.753320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cea5bd-ed17-4875-82b6-e48343ecbb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:50.753974 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cea5bd-ed17-4875-82b6-e48343ecbb77 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:50.754259 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cea5bd-ed17-4875-82b6-e48343ecbb77 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:50.754536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cea5bd-ed17-4875-82b6-e48343ecbb77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:50.755357 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cea5bd-ed17-4875-82b6-e48343ecbb77 None admin] RBAC: Adding query filter params (group.id=6d7d63731f51414185728735a19d856f, role.id=1b988376d12749bd8d653baa89493c5c, scope.project.id=6d7a7a0b84384f14849056212228badb) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:50.755677 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cea5bd-ed17-4875-82b6-e48343ecbb77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:50.757156 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4d1c73-2a0a-44aa-8f1e-8e32a5e0e767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:50.757798 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d1c73-2a0a-44aa-8f1e-8e32a5e0e767 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:50.757896 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cea5bd-ed17-4875-82b6-e48343ecbb77 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:50.758100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d1c73-2a0a-44aa-8f1e-8e32a5e0e767 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:50.758351 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d1c73-2a0a-44aa-8f1e-8e32a5e0e767 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:50.764868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 171/353] 10.222.0.121 () {56 vars in 1381 bytes} [Thu Aug 4 10:18:50 2022] GET /identity/v3/role_assignments?group.id=6d7d63731f51414185728735a19d856f&scope.project.id=6d7a7a0b84384f14849056212228badb&role.id=1b988376d12749bd8d653baa89493c5c => generated 261 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:50.765540 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cea5bd-ed17-4875-82b6-e48343ecbb77 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:50.799478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:50.871345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4d1c73-2a0a-44aa-8f1e-8e32a5e0e767 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:50.873730 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4d1c73-2a0a-44aa-8f1e-8e32a5e0e767 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:50.931803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cea5bd-ed17-4875-82b6-e48343ecbb77 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:50.943570 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:50.981160 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f28ae-ce50-4b99-8bba-bf8b7a6fe075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:50.981866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f28ae-ce50-4b99-8bba-bf8b7a6fe075 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:50.982148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f28ae-ce50-4b99-8bba-bf8b7a6fe075 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:50.982472 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f28ae-ce50-4b99-8bba-bf8b7a6fe075 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:51.018807 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 183/354] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:18:50 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:18:51.020618 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ef2cb-6c40-482d-b4c2-03643bce5f51 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:51.021112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ef2cb-6c40-482d-b4c2-03643bce5f51 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:51.021566 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ef2cb-6c40-482d-b4c2-03643bce5f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:51.042730 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-769ef2cb-6c40-482d-b4c2-03643bce5f51 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:51.044361 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-769ef2cb-6c40-482d-b4c2-03643bce5f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:51.082115 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3f28ae-ce50-4b99-8bba-bf8b7a6fe075 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:51.083023 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3f28ae-ce50-4b99-8bba-bf8b7a6fe075 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:51.108325 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:51.135066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 184/355] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:51 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:51.135790 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ef2cb-6c40-482d-b4c2-03643bce5f51 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:51.221354 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 172/356] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:18:50 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:18:51.222339 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf6ceb-204c-45cd-9cc7-77e0eed366b8 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:51.222614 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf6ceb-204c-45cd-9cc7-77e0eed366b8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:51.222964 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf6ceb-204c-45cd-9cc7-77e0eed366b8 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:51.227504 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 173/357] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:51.227504 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf6ceb-204c-45cd-9cc7-77e0eed366b8 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:51.241517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:51.280112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ef2cb-6c40-482d-b4c2-03643bce5f51 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:51.340305 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc04d61-a269-4792-bc22-6e8e3935f363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:51.342382 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc04d61-a269-4792-bc22-6e8e3935f363 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:51.342510 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc04d61-a269-4792-bc22-6e8e3935f363 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:51.343338 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc04d61-a269-4792-bc22-6e8e3935f363 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:51.366886 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc04d61-a269-4792-bc22-6e8e3935f363 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:51.374000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-6dc04d61-a269-4792-bc22-6e8e3935f363 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:18:51.374000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:51.374000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:51.374000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:51.374000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:51.374000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:51.374000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:51.374000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:51.374000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:51.374000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:51.374000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:51.374000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:51.374000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:51.374000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:51.374000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:51.374000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:51.375004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:51.375004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:51.375004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:51.375004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:51.375004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:51.375004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:51.375004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:51.375004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:51.375004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:51.375004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:51.375004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:51.375004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:51.375004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:51.375004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:51.375004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:51.375004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:51.375854 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:51.375854 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:51.375854 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:51.375854 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:51.375854 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:51.375854 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:51.375854 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:51.375854 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:51.375854 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:51.375854 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:51.375854 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 10:18:51.375854 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:51.375854 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 185/358] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:51.376602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc04d61-a269-4792-bc22-6e8e3935f363 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:51.392155 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf6ceb-204c-45cd-9cc7-77e0eed366b8 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:51.412983 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:51.446070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ed9df-c0ec-4ae6-a3e7-292fd4934f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:51.446772 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ed9df-c0ec-4ae6-a3e7-292fd4934f71 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:51.447054 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ed9df-c0ec-4ae6-a3e7-292fd4934f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:51.447326 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ed9df-c0ec-4ae6-a3e7-292fd4934f71 None admin] PATH_INFO: `/v3/projects/6d7a7a0b84384f14849056212228badb` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:51.449062 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347ed9df-c0ec-4ae6-a3e7-292fd4934f71 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7a7a0b84384f14849056212228badb)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:51.451290 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347ed9df-c0ec-4ae6-a3e7-292fd4934f71 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:51.453529 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 174/359] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:51 2022] GET /identity/v3/projects/6d7a7a0b84384f14849056212228badb => generated 299 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:51.454739 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37064e4d-22ec-4452-8ae5-9f4ca6909c86 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:51.455020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37064e4d-22ec-4452-8ae5-9f4ca6909c86 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:51.455290 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37064e4d-22ec-4452-8ae5-9f4ca6909c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:51.480309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-37064e4d-22ec-4452-8ae5-9f4ca6909c86 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:51.481458 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-37064e4d-22ec-4452-8ae5-9f4ca6909c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:51.524947 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc04d61-a269-4792-bc22-6e8e3935f363 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:51.578612 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612129f-c354-4205-858f-6b3978905260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:51.579413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612129f-c354-4205-858f-6b3978905260 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:51.580343 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612129f-c354-4205-858f-6b3978905260 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:51.580343 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612129f-c354-4205-858f-6b3978905260 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:51.584032 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:51.585072 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 175/360] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:51 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:51.588533 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37064e4d-22ec-4452-8ae5-9f4ca6909c86 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:51.600476 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e612129f-c354-4205-858f-6b3978905260 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:51.606572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-e612129f-c354-4205-858f-6b3978905260 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:18:51.606572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:51.606572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:51.606572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:51.606572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:51.606572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:51.606572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:51.606572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:51.606572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:51.606572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:51.606572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:51.606572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:51.606572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:51.606572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:51.606572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:51.606572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:51.607563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:51.607563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:51.607563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:51.607563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:51.607563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:51.607563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:51.607563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:51.607563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:51.607563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:51.607563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:51.607563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:51.607563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:51.607563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:51.607563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:51.607563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:51.607563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:51.608523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:51.608523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:51.608523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:51.608523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:51.608523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:51.608523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:51.608523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:51.608523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:51.608523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:51.608523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:51.608523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:18:51.608523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:51.611340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 186/361] 10.222.0.121 () {56 vars in 1119 bytes} [Thu Aug 4 10:18:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:51.612563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612129f-c354-4205-858f-6b3978905260 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:51.735993 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37064e4d-22ec-4452-8ae5-9f4ca6909c86 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:51.757716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612129f-c354-4205-858f-6b3978905260 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:51.762865 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:51.788089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e5b9e-d0d2-4596-aa15-f75df4e45f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:51.789164 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e5b9e-d0d2-4596-aa15-f75df4e45f51 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:51.789628 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e5b9e-d0d2-4596-aa15-f75df4e45f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:51.790081 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e5b9e-d0d2-4596-aa15-f75df4e45f51 None admin] PATH_INFO: `/v3/projects/39e472078e7d42f59243230e66c387f8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:51.792482 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132e5b9e-d0d2-4596-aa15-f75df4e45f51 None admin] RBAC: Authorizing `identity:get_project(project_id=39e472078e7d42f59243230e66c387f8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:51.794914 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132e5b9e-d0d2-4596-aa15-f75df4e45f51 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:51.798709 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 176/362] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:51 2022] GET /identity/v3/projects/39e472078e7d42f59243230e66c387f8 => generated 303 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:51.800306 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e5b9e-d0d2-4596-aa15-f75df4e45f51 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:51.806309 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225817e-681e-4faf-afe8-96ec217da9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:51.807087 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225817e-681e-4faf-afe8-96ec217da9f7 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:51.807441 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225817e-681e-4faf-afe8-96ec217da9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:51.807796 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225817e-681e-4faf-afe8-96ec217da9f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:51.808651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d225817e-681e-4faf-afe8-96ec217da9f7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:51.809029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d225817e-681e-4faf-afe8-96ec217da9f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:51.810563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d225817e-681e-4faf-afe8-96ec217da9f7 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:51.821201 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 187/363] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:51 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:51.821918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225817e-681e-4faf-afe8-96ec217da9f7 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:51.904254 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:51.977806 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225817e-681e-4faf-afe8-96ec217da9f7 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:51.993851 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e5b9e-d0d2-4596-aa15-f75df4e45f51 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:52.036031 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9b1dc-e60e-46be-abd5-4449ae6fbc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:52.036730 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9b1dc-e60e-46be-abd5-4449ae6fbc0e None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:52.037011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9b1dc-e60e-46be-abd5-4449ae6fbc0e None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:52.037284 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9b1dc-e60e-46be-abd5-4449ae6fbc0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:52.062115 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb022ac7-665e-4cfe-9672-0bdb32391e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:52.062953 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb022ac7-665e-4cfe-9672-0bdb32391e33 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:52.063124 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb022ac7-665e-4cfe-9672-0bdb32391e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:52.063365 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb022ac7-665e-4cfe-9672-0bdb32391e33 None admin] PATH_INFO: `/v3/groups/6d7d63731f51414185728735a19d856f` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:52.065010 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9b1dc-e60e-46be-abd5-4449ae6fbc0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:52.065135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb022ac7-665e-4cfe-9672-0bdb32391e33 None admin] RBAC: Authorizing `identity:get_group(group_id=6d7d63731f51414185728735a19d856f)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:52.066584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb022ac7-665e-4cfe-9672-0bdb32391e33 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:52.068768 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 177/364] 10.222.0.121 () {56 vars in 1185 bytes} [Thu Aug 4 10:18:51 2022] GET /identity/v3/groups/6d7d63731f51414185728735a19d856f => generated 227 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:52.069616 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb022ac7-665e-4cfe-9672-0bdb32391e33 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:52.076887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-bec9b1dc-e60e-46be-abd5-4449ae6fbc0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:18:52.076887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:52.076887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:52.076887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:52.076887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:52.076887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:52.076887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:52.076887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:52.076887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:52.076887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:52.076887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:52.076887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:52.076887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:52.076887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:52.076887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:52.076887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:52.077909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:52.077909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:52.077909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:52.077909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:52.077909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:52.077909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:52.077909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:52.077909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:52.077909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:52.077909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:52.077909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:52.077909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:52.077909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:52.077909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:52.077909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:52.077909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:52.079008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:52.079008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:52.079008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:52.079008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:52.079008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:52.079008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:52.079008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:52.079008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:52.079008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:52.079008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:52.079008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:18:52.079008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:52.079008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 188/365] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:18:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:52.079712 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9b1dc-e60e-46be-abd5-4449ae6fbc0e None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:52.162869 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:52.241507 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb022ac7-665e-4cfe-9672-0bdb32391e33 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:52.265340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9b1dc-e60e-46be-abd5-4449ae6fbc0e None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:52.284248 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e139cee-c61a-4d6e-b5ac-77c4d99e8612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:52.284903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e139cee-c61a-4d6e-b5ac-77c4d99e8612 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:52.285188 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e139cee-c61a-4d6e-b5ac-77c4d99e8612 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:52.285459 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e139cee-c61a-4d6e-b5ac-77c4d99e8612 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:52.286204 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e139cee-c61a-4d6e-b5ac-77c4d99e8612 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:52.286511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e139cee-c61a-4d6e-b5ac-77c4d99e8612 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:52.289945 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e139cee-c61a-4d6e-b5ac-77c4d99e8612 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:52.292180 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:52.299252 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 178/366] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:18:52 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:52.300181 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e139cee-c61a-4d6e-b5ac-77c4d99e8612 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:52.331983 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7759247-5614-4ad4-a6ca-8fe2c6f8e705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:52.331983 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7759247-5614-4ad4-a6ca-8fe2c6f8e705 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:52.331983 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7759247-5614-4ad4-a6ca-8fe2c6f8e705 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:52.331983 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7759247-5614-4ad4-a6ca-8fe2c6f8e705 None admin] PATH_INFO: `/v3/groups/6d7d63731f51414185728735a19d856f` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:52.335969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7759247-5614-4ad4-a6ca-8fe2c6f8e705 None admin] RBAC: Authorizing `identity:get_group(group_id=6d7d63731f51414185728735a19d856f)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:52.335969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7759247-5614-4ad4-a6ca-8fe2c6f8e705 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:52.343983 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 189/367] 10.222.0.121 () {56 vars in 1185 bytes} [Thu Aug 4 10:18:52 2022] GET /identity/v3/groups/6d7d63731f51414185728735a19d856f => generated 227 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:52.343983 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7759247-5614-4ad4-a6ca-8fe2c6f8e705 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:52.439678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e139cee-c61a-4d6e-b5ac-77c4d99e8612 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:52.456531 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:52.484621 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5502832b-b5c3-4392-a066-bfe88334aacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:52.485302 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502832b-b5c3-4392-a066-bfe88334aacc None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:52.485587 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502832b-b5c3-4392-a066-bfe88334aacc None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:52.485864 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502832b-b5c3-4392-a066-bfe88334aacc None admin] PATH_INFO: `/v3/users/72217dde07a54e9bad663fa3db183281` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:52.487487 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5502832b-b5c3-4392-a066-bfe88334aacc None admin] RBAC: Authorizing `identity:get_user(user_id=72217dde07a54e9bad663fa3db183281)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:52.489736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5502832b-b5c3-4392-a066-bfe88334aacc None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:52.490876 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7759247-5614-4ad4-a6ca-8fe2c6f8e705 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:52.492763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 179/368] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:18:52 2022] GET /identity/v3/users/72217dde07a54e9bad663fa3db183281 => generated 294 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:52.493629 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5502832b-b5c3-4392-a066-bfe88334aacc None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:52.551626 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50289f47-a42d-4907-bc44-bc3d9166fea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:52.552371 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50289f47-a42d-4907-bc44-bc3d9166fea6 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:52.552727 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50289f47-a42d-4907-bc44-bc3d9166fea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:52.552727 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50289f47-a42d-4907-bc44-bc3d9166fea6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:52.552727 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50289f47-a42d-4907-bc44-bc3d9166fea6 None admin] RBAC: Adding query filter params (group.id=6d7d63731f51414185728735a19d856f, role.id=89b847fca3574b9083287279aee0f773, scope.project.id=6d7a7a0b84384f14849056212228badb) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:52.552727 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50289f47-a42d-4907-bc44-bc3d9166fea6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:52.555709 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50289f47-a42d-4907-bc44-bc3d9166fea6 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:52.564239 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 190/369] 10.222.0.121 () {56 vars in 1381 bytes} [Thu Aug 4 10:18:52 2022] GET /identity/v3/role_assignments?group.id=6d7d63731f51414185728735a19d856f&scope.project.id=6d7a7a0b84384f14849056212228badb&role.id=89b847fca3574b9083287279aee0f773 => generated 261 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:52.565140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50289f47-a42d-4907-bc44-bc3d9166fea6 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:52.662315 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:52.689244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5502832b-b5c3-4392-a066-bfe88334aacc None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:52.746304 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b1e44-cc50-46d8-8005-92ab9abc4871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:52.747467 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b1e44-cc50-46d8-8005-92ab9abc4871 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:52.747467 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b1e44-cc50-46d8-8005-92ab9abc4871 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:52.747467 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b1e44-cc50-46d8-8005-92ab9abc4871 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:52.748392 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b1e44-cc50-46d8-8005-92ab9abc4871 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:52.748708 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b1e44-cc50-46d8-8005-92ab9abc4871 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:52.750211 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b1e44-cc50-46d8-8005-92ab9abc4871 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:52.766210 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 180/370] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:52 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:52.767353 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b1e44-cc50-46d8-8005-92ab9abc4871 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:52.774591 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50289f47-a42d-4907-bc44-bc3d9166fea6 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:52.825634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d5267-ed98-4392-8b0f-516029f014d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:52.826653 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d5267-ed98-4392-8b0f-516029f014d5 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:52.827391 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d5267-ed98-4392-8b0f-516029f014d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:52.827824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d5267-ed98-4392-8b0f-516029f014d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:52.840389 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:52.901826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b1e44-cc50-46d8-8005-92ab9abc4871 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:52.935640 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d5267-ed98-4392-8b0f-516029f014d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:52.937315 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d5267-ed98-4392-8b0f-516029f014d5 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:52.957692 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cf65b-40cd-44d8-bd25-87317766ba82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:52.958566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cf65b-40cd-44d8-bd25-87317766ba82 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:52.958961 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cf65b-40cd-44d8-bd25-87317766ba82 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:52.959348 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cf65b-40cd-44d8-bd25-87317766ba82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:52.960422 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513cf65b-40cd-44d8-bd25-87317766ba82 None admin] RBAC: Adding query filter params (group.id=6d7d63731f51414185728735a19d856f, role.id=1b988376d12749bd8d653baa89493c5c, scope.project.id=39e472078e7d42f59243230e66c387f8) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:52.960861 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513cf65b-40cd-44d8-bd25-87317766ba82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:52.963486 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513cf65b-40cd-44d8-bd25-87317766ba82 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:52.976051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 181/371] 10.222.0.121 () {56 vars in 1381 bytes} [Thu Aug 4 10:18:52 2022] GET /identity/v3/role_assignments?group.id=6d7d63731f51414185728735a19d856f&scope.project.id=39e472078e7d42f59243230e66c387f8&role.id=1b988376d12749bd8d653baa89493c5c => generated 261 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:52.976051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cf65b-40cd-44d8-bd25-87317766ba82 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:53.026231 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:53.117295 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cf65b-40cd-44d8-bd25-87317766ba82 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:53.117501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 191/372] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:18:52 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:18:53.118382 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d5267-ed98-4392-8b0f-516029f014d5 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:53.138338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:53.187797 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6adc4-3bd1-4393-99ec-445110c7ce7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:53.188512 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6adc4-3bd1-4393-99ec-445110c7ce7e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:53.188807 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6adc4-3bd1-4393-99ec-445110c7ce7e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:53.189090 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6adc4-3bd1-4393-99ec-445110c7ce7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:53.189907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6adc4-3bd1-4393-99ec-445110c7ce7e None admin] RBAC: Adding query filter params (role.id=c77f1c570e8640a3b2c836a401b91889, user.id=72217dde07a54e9bad663fa3db183281, scope.system=all) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:53.190225 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6adc4-3bd1-4393-99ec-445110c7ce7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:53.191708 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6adc4-3bd1-4393-99ec-445110c7ce7e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:53.209083 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 182/373] 10.222.0.121 () {56 vars in 1313 bytes} [Thu Aug 4 10:18:52 2022] GET /identity/v3/role_assignments?user.id=72217dde07a54e9bad663fa3db183281&scope.system=all&role.id=c77f1c570e8640a3b2c836a401b91889 => generated 512 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:53.210505 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f4c32-75a5-4405-bfbc-3d30145bce7a None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:53.210799 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f4c32-75a5-4405-bfbc-3d30145bce7a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:53.211080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f4c32-75a5-4405-bfbc-3d30145bce7a None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:53.212952 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 183/374] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:53.213774 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f4c32-75a5-4405-bfbc-3d30145bce7a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:53.325737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d5267-ed98-4392-8b0f-516029f014d5 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:53.392219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159f4be-ae8c-4695-9c3f-d33448c3a41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:53.392831 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159f4be-ae8c-4695-9c3f-d33448c3a41b None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:53.393114 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159f4be-ae8c-4695-9c3f-d33448c3a41b None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:53.393387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159f4be-ae8c-4695-9c3f-d33448c3a41b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:53.416359 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:53.420616 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f4c32-75a5-4405-bfbc-3d30145bce7a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:53.465118 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f38ceb-24b6-48a9-95c0-774c21fb4047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:53.466039 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f38ceb-24b6-48a9-95c0-774c21fb4047 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:53.466323 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f38ceb-24b6-48a9-95c0-774c21fb4047 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:53.466599 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f38ceb-24b6-48a9-95c0-774c21fb4047 None admin] PATH_INFO: `/v3/projects/39e472078e7d42f59243230e66c387f8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:53.467830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f38ceb-24b6-48a9-95c0-774c21fb4047 None admin] RBAC: Authorizing `identity:get_project(project_id=39e472078e7d42f59243230e66c387f8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:53.470055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f38ceb-24b6-48a9-95c0-774c21fb4047 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:53.472658 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 184/375] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:53 2022] GET /identity/v3/projects/39e472078e7d42f59243230e66c387f8 => generated 303 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:53.474602 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d2e3a-60c4-4d46-bf3f-f9326e5e95e5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:53.475011 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d2e3a-60c4-4d46-bf3f-f9326e5e95e5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:53.475332 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d2e3a-60c4-4d46-bf3f-f9326e5e95e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:53.481018 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6159f4be-ae8c-4695-9c3f-d33448c3a41b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:53.482563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6159f4be-ae8c-4695-9c3f-d33448c3a41b None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:53.500019 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-022d2e3a-60c4-4d46-bf3f-f9326e5e95e5 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:53.501320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-022d2e3a-60c4-4d46-bf3f-f9326e5e95e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:53.539197 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:53.604171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 185/376] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:53 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:53.605493 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d2e3a-60c4-4d46-bf3f-f9326e5e95e5 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:53.654458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 192/377] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:18:53 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:18:53.654806 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159f4be-ae8c-4695-9c3f-d33448c3a41b None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:53.717285 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:53.732060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d2e3a-60c4-4d46-bf3f-f9326e5e95e5 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:53.771185 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec59cc4-7eee-4744-a631-9eeb23d8c744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:53.771931 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec59cc4-7eee-4744-a631-9eeb23d8c744 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:53.772324 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec59cc4-7eee-4744-a631-9eeb23d8c744 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:53.772516 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec59cc4-7eee-4744-a631-9eeb23d8c744 None admin] PATH_INFO: `/v3/groups/6d7d63731f51414185728735a19d856f` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:53.773819 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec59cc4-7eee-4744-a631-9eeb23d8c744 None admin] RBAC: Authorizing `identity:get_group(group_id=6d7d63731f51414185728735a19d856f)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:53.775298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec59cc4-7eee-4744-a631-9eeb23d8c744 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:53.777390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 186/378] 10.222.0.121 () {56 vars in 1185 bytes} [Thu Aug 4 10:18:53 2022] GET /identity/v3/groups/6d7d63731f51414185728735a19d856f => generated 227 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:53.778303 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec59cc4-7eee-4744-a631-9eeb23d8c744 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:53.782219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159f4be-ae8c-4695-9c3f-d33448c3a41b None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:53.812830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:53.825989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bff8dbc-7d99-4423-8670-33177ef51119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:53.827134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff8dbc-7d99-4423-8670-33177ef51119 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:53.828353 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff8dbc-7d99-4423-8670-33177ef51119 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:53.828878 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff8dbc-7d99-4423-8670-33177ef51119 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:53.846844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bff8dbc-7d99-4423-8670-33177ef51119 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:53.855240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-9bff8dbc-7d99-4423-8670-33177ef51119 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:53.855240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:53.855240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:53.855240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:53.855240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:53.855240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:53.855240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:53.855240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:53.855240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:53.855240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:53.855240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:53.855240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:53.855240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:53.855240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:53.855240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:53.855240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:53.856310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:53.856310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:53.856310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:53.856310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:53.856310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:53.856310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:53.856310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:53.856310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:53.856310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:53.856310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:53.856310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:53.856310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:53.856310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:53.856310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:53.856310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:53.856310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:53.857135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:53.857135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:53.857135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:53.857135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:53.857135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:53.857135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:53.857135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:53.857135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:53.857135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:53.857135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:53.857135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:53.857135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:53.858390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 193/379] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:53.858865 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bff8dbc-7d99-4423-8670-33177ef51119 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:53.915193 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec59cc4-7eee-4744-a631-9eeb23d8c744 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:53.970506 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c0d40-ad0b-46ef-9af8-87a5c5ab93fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:53.971047 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c0d40-ad0b-46ef-9af8-87a5c5ab93fd None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:53.971333 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c0d40-ad0b-46ef-9af8-87a5c5ab93fd None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:53.971610 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c0d40-ad0b-46ef-9af8-87a5c5ab93fd None admin] PATH_INFO: `/v3/projects/03f48ab98257418c836fe2e290bdf749` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:53.976874 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c0d40-ad0b-46ef-9af8-87a5c5ab93fd None admin] RBAC: Authorizing `identity:get_project(project_id=03f48ab98257418c836fe2e290bdf749)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:53.978447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c0d40-ad0b-46ef-9af8-87a5c5ab93fd None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:53.981821 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 187/380] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:53 2022] GET /identity/v3/projects/03f48ab98257418c836fe2e290bdf749 => generated 345 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:53.982756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c0d40-ad0b-46ef-9af8-87a5c5ab93fd None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:54.022562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bff8dbc-7d99-4423-8670-33177ef51119 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:54.037963 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:54.076640 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249aad4-31c3-41c0-99da-4fd0b7adb8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:54.077391 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249aad4-31c3-41c0-99da-4fd0b7adb8ac None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:54.077742 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249aad4-31c3-41c0-99da-4fd0b7adb8ac None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:54.078097 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249aad4-31c3-41c0-99da-4fd0b7adb8ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:54.079122 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5249aad4-31c3-41c0-99da-4fd0b7adb8ac None admin] RBAC: Adding query filter params (group.id=6d7d63731f51414185728735a19d856f, role.id=89b847fca3574b9083287279aee0f773, scope.project.id=39e472078e7d42f59243230e66c387f8) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:54.079303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5249aad4-31c3-41c0-99da-4fd0b7adb8ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:54.081102 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5249aad4-31c3-41c0-99da-4fd0b7adb8ac None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:54.093569 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 194/381] 10.222.0.121 () {56 vars in 1381 bytes} [Thu Aug 4 10:18:53 2022] GET /identity/v3/role_assignments?group.id=6d7d63731f51414185728735a19d856f&scope.project.id=39e472078e7d42f59243230e66c387f8&role.id=89b847fca3574b9083287279aee0f773 => generated 261 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:54.095295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249aad4-31c3-41c0-99da-4fd0b7adb8ac None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:54.138487 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:54.142487 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c0d40-ad0b-46ef-9af8-87a5c5ab93fd None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:54.210083 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608d09e-543e-4e77-b3a7-0e6bcfaeaeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:54.210747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608d09e-543e-4e77-b3a7-0e6bcfaeaeb0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:54.211102 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608d09e-543e-4e77-b3a7-0e6bcfaeaeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:54.211475 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608d09e-543e-4e77-b3a7-0e6bcfaeaeb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:54.212537 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608d09e-543e-4e77-b3a7-0e6bcfaeaeb0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:54.212919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608d09e-543e-4e77-b3a7-0e6bcfaeaeb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:54.214453 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608d09e-543e-4e77-b3a7-0e6bcfaeaeb0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:54.228389 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 188/382] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:53 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:54.237506 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608d09e-543e-4e77-b3a7-0e6bcfaeaeb0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:54.296090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249aad4-31c3-41c0-99da-4fd0b7adb8ac None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:54.377118 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a362e-4d2e-4a76-bf00-f5bf277bddaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:54.378022 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a362e-4d2e-4a76-bf00-f5bf277bddaf None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:54.378222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a362e-4d2e-4a76-bf00-f5bf277bddaf None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:54.378366 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a362e-4d2e-4a76-bf00-f5bf277bddaf None admin] PATH_INFO: `/v3/groups/9c252f85b3564ef3a9bc3ac49cd07405` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:54.390826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a362e-4d2e-4a76-bf00-f5bf277bddaf None admin] RBAC: Authorizing `identity:get_group(group_id=9c252f85b3564ef3a9bc3ac49cd07405)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:54.392442 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a362e-4d2e-4a76-bf00-f5bf277bddaf None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:54.394254 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 195/383] 10.222.0.121 () {56 vars in 1185 bytes} [Thu Aug 4 10:18:54 2022] GET /identity/v3/groups/9c252f85b3564ef3a9bc3ac49cd07405 => generated 230 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:54.400750 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a362e-4d2e-4a76-bf00-f5bf277bddaf None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:54.480643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:54.486578 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608d09e-543e-4e77-b3a7-0e6bcfaeaeb0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:54.556504 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a23948-43e0-4506-b01f-1db8cb84c043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:54.557182 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a23948-43e0-4506-b01f-1db8cb84c043 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:54.557467 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a23948-43e0-4506-b01f-1db8cb84c043 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:54.557749 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a23948-43e0-4506-b01f-1db8cb84c043 None admin] PATH_INFO: `/v3/groups/6d7d63731f51414185728735a19d856f` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:54.559623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a23948-43e0-4506-b01f-1db8cb84c043 None admin] RBAC: Authorizing `identity:get_group(group_id=6d7d63731f51414185728735a19d856f)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:54.561163 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a23948-43e0-4506-b01f-1db8cb84c043 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:54.565014 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 189/384] 10.222.0.121 () {56 vars in 1185 bytes} [Thu Aug 4 10:18:54 2022] GET /identity/v3/groups/6d7d63731f51414185728735a19d856f => generated 227 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:54.568666 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a23948-43e0-4506-b01f-1db8cb84c043 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:54.604717 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a362e-4d2e-4a76-bf00-f5bf277bddaf None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:54.653520 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:54.703472 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a23948-43e0-4506-b01f-1db8cb84c043 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:54.724884 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a74159-c14c-4815-a8c8-8c016cd7cdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:54.725663 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a74159-c14c-4815-a8c8-8c016cd7cdfe None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:54.726003 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a74159-c14c-4815-a8c8-8c016cd7cdfe None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:54.726287 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a74159-c14c-4815-a8c8-8c016cd7cdfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:54.727116 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a74159-c14c-4815-a8c8-8c016cd7cdfe None admin] RBAC: Adding query filter params (group.id=9c252f85b3564ef3a9bc3ac49cd07405, role.id=7342106a255b4bce8f150726602a8b95, scope.project.id=03f48ab98257418c836fe2e290bdf749) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:54.727433 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a74159-c14c-4815-a8c8-8c016cd7cdfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:54.729069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a74159-c14c-4815-a8c8-8c016cd7cdfe None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:54.740308 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025be17-f7c0-494e-89e7-287541244fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:54.740959 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025be17-f7c0-494e-89e7-287541244fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:54.741256 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025be17-f7c0-494e-89e7-287541244fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:54.741536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025be17-f7c0-494e-89e7-287541244fb7 None admin] PATH_INFO: `/v3/projects/6d7a7a0b84384f14849056212228badb` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:54.742800 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025be17-f7c0-494e-89e7-287541244fb7 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7a7a0b84384f14849056212228badb)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:54.744309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025be17-f7c0-494e-89e7-287541244fb7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:54.746380 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 190/385] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:54 2022] GET /identity/v3/projects/6d7a7a0b84384f14849056212228badb => generated 299 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:54.747143 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 196/386] 10.222.0.121 () {56 vars in 1381 bytes} [Thu Aug 4 10:18:54 2022] GET /identity/v3/role_assignments?group.id=9c252f85b3564ef3a9bc3ac49cd07405&scope.project.id=03f48ab98257418c836fe2e290bdf749&role.id=7342106a255b4bce8f150726602a8b95 => generated 261 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:18:54.752526 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025be17-f7c0-494e-89e7-287541244fb7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:54.896125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:54.903058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025be17-f7c0-494e-89e7-287541244fb7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:54.946547 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da838a7-27ee-44fe-94d8-0ae2fd021366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:54.947319 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da838a7-27ee-44fe-94d8-0ae2fd021366 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:54.947604 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da838a7-27ee-44fe-94d8-0ae2fd021366 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:54.947889 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da838a7-27ee-44fe-94d8-0ae2fd021366 None admin] PATH_INFO: `/v3/projects/6d7a7a0b84384f14849056212228badb/groups/6d7d63731f51414185728735a19d856f/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:54.950064 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da838a7-27ee-44fe-94d8-0ae2fd021366 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d7a7a0b84384f14849056212228badb, group_id=6d7d63731f51414185728735a19d856f, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:54.951563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da838a7-27ee-44fe-94d8-0ae2fd021366 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:54.972177 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 191/387] 10.222.0.121 () {58 vars in 1447 bytes} [Thu Aug 4 10:18:54 2022] PUT /identity/v3/projects/6d7a7a0b84384f14849056212228badb/groups/6d7d63731f51414185728735a19d856f/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:18:55.411849 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b6280-c714-44a6-b995-b319ceae2db5 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:55.412768 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b6280-c714-44a6-b995-b319ceae2db5 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:55.412768 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b6280-c714-44a6-b995-b319ceae2db5 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:55.414283 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 197/388] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:55 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:55.420575 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b99075-a9ca-4c22-91f7-d63327443935 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:55.420879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b99075-a9ca-4c22-91f7-d63327443935 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:55.421149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b99075-a9ca-4c22-91f7-d63327443935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:55.453133 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d2b99075-a9ca-4c22-91f7-d63327443935 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:55.454403 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d2b99075-a9ca-4c22-91f7-d63327443935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:55.543370 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 192/389] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:55 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:55.572227 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b6280-c714-44a6-b995-b319ceae2db5 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:55.707114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f5b5a-6bf5-454d-afc9-0ca2afb10d36 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:55.707347 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f5b5a-6bf5-454d-afc9-0ca2afb10d36 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:55.707653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f5b5a-6bf5-454d-afc9-0ca2afb10d36 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:55.709202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 193/390] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:55 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:55.720148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b2219-ed01-4384-86c6-f0076f5bf827 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:55.720872 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b2219-ed01-4384-86c6-f0076f5bf827 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:55.721344 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b2219-ed01-4384-86c6-f0076f5bf827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:55.754424 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:55.754662 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-642b2219-ed01-4384-86c6-f0076f5bf827 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:55.757522 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b6280-c714-44a6-b995-b319ceae2db5 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:55.760214 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-642b2219-ed01-4384-86c6-f0076f5bf827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:55.830826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99515ab-c76e-47b2-8851-b687f61f4be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:55.831513 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99515ab-c76e-47b2-8851-b687f61f4be6 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:55.831799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99515ab-c76e-47b2-8851-b687f61f4be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:55.832127 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99515ab-c76e-47b2-8851-b687f61f4be6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:55.855235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99515ab-c76e-47b2-8851-b687f61f4be6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:55.861596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-f99515ab-c76e-47b2-8851-b687f61f4be6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:18:55.861596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:55.861596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:55.861596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:55.861596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:55.861596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:55.861596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:55.861596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:55.861596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:55.861596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:55.861596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:55.861596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:55.861596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:55.861596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:55.861596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:55.861596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:55.862572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:55.862572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:55.862572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:55.862572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:55.862572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:55.862572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:55.862572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:55.862572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:55.862572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:55.862572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:55.862572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:55.862572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:55.862572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:55.862572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:55.862572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:55.862572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:55.863752 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:55.863752 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:55.863752 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:55.863752 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:55.863752 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:55.863752 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:55.863752 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:55.863752 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:55.863752 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:55.863752 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:55.863752 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:18:55.863752 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:55.865476 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 198/391] 10.222.0.121 () {56 vars in 1119 bytes} [Thu Aug 4 10:18:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:55.872248 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99515ab-c76e-47b2-8851-b687f61f4be6 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:55.907159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 194/392] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:55 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:55.933176 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b2219-ed01-4384-86c6-f0076f5bf827 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:55.969603 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:56.059004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99515ab-c76e-47b2-8851-b687f61f4be6 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:56.127169 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec29dc6-b99b-4272-9d80-6171d0e935a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:56.127831 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec29dc6-b99b-4272-9d80-6171d0e935a1 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:56.128212 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec29dc6-b99b-4272-9d80-6171d0e935a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:56.128484 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec29dc6-b99b-4272-9d80-6171d0e935a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:56.129229 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec29dc6-b99b-4272-9d80-6171d0e935a1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:56.129533 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec29dc6-b99b-4272-9d80-6171d0e935a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:56.130938 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec29dc6-b99b-4272-9d80-6171d0e935a1 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:56.144251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b2219-ed01-4384-86c6-f0076f5bf827 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:56.150114 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 199/393] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:18:55 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:56.156286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec29dc6-b99b-4272-9d80-6171d0e935a1 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:56.188325 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:56.261957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1be72-656a-4a97-a353-75575a88df3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:56.262535 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1be72-656a-4a97-a353-75575a88df3d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:56.262825 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1be72-656a-4a97-a353-75575a88df3d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:56.263102 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1be72-656a-4a97-a353-75575a88df3d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:56.286639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec29dc6-b99b-4272-9d80-6171d0e935a1 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:56.300697 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a1be72-656a-4a97-a353-75575a88df3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:56.305866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-a1a1be72-656a-4a97-a353-75575a88df3d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:56.305866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:56.305866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:56.305866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:56.305866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:56.305866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:56.305866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:56.305866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:56.305866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:56.305866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:56.305866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:56.305866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:56.305866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:56.305866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:56.305866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:56.305866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:56.306838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:56.306838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:56.306838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:56.306838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:56.306838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:56.306838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:56.306838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:56.306838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:56.306838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:56.306838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:56.306838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:56.306838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:56.306838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:56.306838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:56.306838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:56.306838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:56.307684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:56.307684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:56.307684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:56.307684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:56.307684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:56.307684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:56.307684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:56.307684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:56.307684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:56.307684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:56.307684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:56.307684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:18:56.307684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 195/394] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:56.316709 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1be72-656a-4a97-a353-75575a88df3d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:56.331980 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c5e08-bc63-4281-9f11-7c4b5e9f4d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:56.332721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c5e08-bc63-4281-9f11-7c4b5e9f4d67 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:56.333002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c5e08-bc63-4281-9f11-7c4b5e9f4d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:56.333276 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c5e08-bc63-4281-9f11-7c4b5e9f4d67 None admin] PATH_INFO: `/v3/groups/6d7d63731f51414185728735a19d856f` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:56.334520 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c5e08-bc63-4281-9f11-7c4b5e9f4d67 None admin] RBAC: Authorizing `identity:get_group(group_id=6d7d63731f51414185728735a19d856f)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:56.336738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c5e08-bc63-4281-9f11-7c4b5e9f4d67 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:56.338759 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 200/395] 10.222.0.121 () {56 vars in 1185 bytes} [Thu Aug 4 10:18:56 2022] GET /identity/v3/groups/6d7d63731f51414185728735a19d856f => generated 227 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:18:56.344953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c5e08-bc63-4281-9f11-7c4b5e9f4d67 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:56.515412 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:56.567504 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:56.577304 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1be72-656a-4a97-a353-75575a88df3d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:56.592020 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c5e08-bc63-4281-9f11-7c4b5e9f4d67 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:56.636709 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133b3db-17d8-40c2-815a-e4bc3ff379b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:56.637446 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133b3db-17d8-40c2-815a-e4bc3ff379b9 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:56.637727 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133b3db-17d8-40c2-815a-e4bc3ff379b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:56.637998 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133b3db-17d8-40c2-815a-e4bc3ff379b9 None admin] PATH_INFO: `/v3/projects/6d7a7a0b84384f14849056212228badb` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:56.640180 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133b3db-17d8-40c2-815a-e4bc3ff379b9 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7a7a0b84384f14849056212228badb)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:56.641735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133b3db-17d8-40c2-815a-e4bc3ff379b9 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:56.644147 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 201/396] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:56 2022] GET /identity/v3/projects/6d7a7a0b84384f14849056212228badb => generated 299 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:56.653472 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133b3db-17d8-40c2-815a-e4bc3ff379b9 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:56.681456 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9718ca-2718-465a-8ba3-7cc45f462212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:56.682110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9718ca-2718-465a-8ba3-7cc45f462212 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:56.682393 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9718ca-2718-465a-8ba3-7cc45f462212 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:56.682683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9718ca-2718-465a-8ba3-7cc45f462212 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:56.683432 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9718ca-2718-465a-8ba3-7cc45f462212 None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:56.683747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9718ca-2718-465a-8ba3-7cc45f462212 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:56.685561 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9718ca-2718-465a-8ba3-7cc45f462212 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:56.707860 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 196/397] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:56 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:56.714029 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9718ca-2718-465a-8ba3-7cc45f462212 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:56.868059 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133b3db-17d8-40c2-815a-e4bc3ff379b9 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:56.890508 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9718ca-2718-465a-8ba3-7cc45f462212 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:56.914518 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7c160-b0f0-4db4-9fb2-35b93bab23a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:56.915798 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7c160-b0f0-4db4-9fb2-35b93bab23a6 None admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:56.916148 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7c160-b0f0-4db4-9fb2-35b93bab23a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:56.916439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7c160-b0f0-4db4-9fb2-35b93bab23a6 None admin] PATH_INFO: `/v3/projects/6d7a7a0b84384f14849056212228badb/groups/6d7d63731f51414185728735a19d856f/roles/89b847fca3574b9083287279aee0f773` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:56.919362 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7c160-b0f0-4db4-9fb2-35b93bab23a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d7a7a0b84384f14849056212228badb, group_id=6d7d63731f51414185728735a19d856f, role_id=89b847fca3574b9083287279aee0f773)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:56.921569 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7c160-b0f0-4db4-9fb2-35b93bab23a6 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:56.940788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 202/398] 10.222.0.121 () {58 vars in 1447 bytes} [Thu Aug 4 10:18:56 2022] PUT /identity/v3/projects/6d7a7a0b84384f14849056212228badb/groups/6d7d63731f51414185728735a19d856f/roles/89b847fca3574b9083287279aee0f773 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:18:56.942597 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b1f83-2610-44eb-9b53-ac99192a48fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:56.943345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b1f83-2610-44eb-9b53-ac99192a48fc None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:56.943744 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b1f83-2610-44eb-9b53-ac99192a48fc None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:56.944316 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b1f83-2610-44eb-9b53-ac99192a48fc None admin] PATH_INFO: `/v3/groups/6d7d63731f51414185728735a19d856f` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:56.946237 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63b1f83-2610-44eb-9b53-ac99192a48fc None admin] RBAC: Authorizing `identity:get_group(group_id=6d7d63731f51414185728735a19d856f)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:56.947749 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63b1f83-2610-44eb-9b53-ac99192a48fc None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:56.949457 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fdbde-e226-47b0-984a-e1796a886c7a None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:56.950218 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fdbde-e226-47b0-984a-e1796a886c7a None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:56.950913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fdbde-e226-47b0-984a-e1796a886c7a None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:56.960028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 197/400] 10.222.0.121 () {56 vars in 1185 bytes} [Thu Aug 4 10:18:56 2022] GET /identity/v3/groups/6d7d63731f51414185728735a19d856f => generated 227 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:18:56.960028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b1f83-2610-44eb-9b53-ac99192a48fc None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:56.960764 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 203/400] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:56 2022] GET /identity => generated 271 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:56.970232 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa29fc-2ccc-42c6-ac7b-5e20f7b9210b None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:56.971153 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa29fc-2ccc-42c6-ac7b-5e20f7b9210b None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:56.971867 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa29fc-2ccc-42c6-ac7b-5e20f7b9210b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:56.978136 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:57.018904 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-bfaa29fc-2ccc-42c6-ac7b-5e20f7b9210b None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:57.020283 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-bfaa29fc-2ccc-42c6-ac7b-5e20f7b9210b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:57.058167 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:57.154140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 204/401] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:56 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:57.169513 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b1f83-2610-44eb-9b53-ac99192a48fc None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:57.182334 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa29fc-2ccc-42c6-ac7b-5e20f7b9210b None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:57.215192 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39356310-ac75-4d5c-b9a5-8a7e782fc8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:57.216366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39356310-ac75-4d5c-b9a5-8a7e782fc8da None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:57.216868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39356310-ac75-4d5c-b9a5-8a7e782fc8da None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:57.217407 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39356310-ac75-4d5c-b9a5-8a7e782fc8da None admin] PATH_INFO: `/v3/projects/39e472078e7d42f59243230e66c387f8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:57.219232 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39356310-ac75-4d5c-b9a5-8a7e782fc8da None admin] RBAC: Authorizing `identity:get_project(project_id=39e472078e7d42f59243230e66c387f8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:57.221177 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39356310-ac75-4d5c-b9a5-8a7e782fc8da None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:57.224429 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 198/402] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:56 2022] GET /identity/v3/projects/39e472078e7d42f59243230e66c387f8 => generated 303 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:57.226212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11b9e6-d426-4d4a-9ef2-5600d42c7c72 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:57.226692 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11b9e6-d426-4d4a-9ef2-5600d42c7c72 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:57.227119 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11b9e6-d426-4d4a-9ef2-5600d42c7c72 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:57.229279 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 199/403] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:57 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:57.230973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11b9e6-d426-4d4a-9ef2-5600d42c7c72 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:57.326614 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa29fc-2ccc-42c6-ac7b-5e20f7b9210b None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:57.378362 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:57.380211 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784a3af-ffcf-4f0f-a605-287b0b1b7ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:57.381211 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784a3af-ffcf-4f0f-a605-287b0b1b7ffc None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:57.381853 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784a3af-ffcf-4f0f-a605-287b0b1b7ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:57.382333 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784a3af-ffcf-4f0f-a605-287b0b1b7ffc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:57.395449 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3784a3af-ffcf-4f0f-a605-287b0b1b7ffc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:57.401220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-3784a3af-ffcf-4f0f-a605-287b0b1b7ffc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:18:57.401220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:57.401220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:57.401220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:57.401220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:57.401220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:57.401220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:57.401220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:57.401220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:57.401220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:57.401220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:57.401220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:57.401220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:57.401220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:57.401220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:57.401220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:57.402214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:57.402214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:57.402214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:57.402214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:57.402214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:57.402214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:57.402214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:57.402214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:57.402214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:57.402214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:57.402214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:57.402214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:57.402214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:57.402214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:57.402214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:57.402214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:57.403157 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:57.403157 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:57.403157 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:57.403157 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:57.403157 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:57.403157 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:57.403157 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:57.403157 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:57.403157 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:57.403157 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:57.403157 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:18:57.403157 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:57.406259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 205/404] 10.222.0.121 () {56 vars in 1119 bytes} [Thu Aug 4 10:18:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:57.408546 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ca23a-1d1b-425c-8b13-0ebb56c6799d None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:57.409085 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ca23a-1d1b-425c-8b13-0ebb56c6799d None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:57.409617 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ca23a-1d1b-425c-8b13-0ebb56c6799d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:57.412746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11b9e6-d426-4d4a-9ef2-5600d42c7c72 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:57.430231 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-e32ca23a-1d1b-425c-8b13-0ebb56c6799d None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:57.432002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-e32ca23a-1d1b-425c-8b13-0ebb56c6799d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:57.434880 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:57.460753 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6ff81-fa32-4381-94c5-9d030e4464a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:57.461678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6ff81-fa32-4381-94c5-9d030e4464a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:57.461962 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6ff81-fa32-4381-94c5-9d030e4464a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:57.462241 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6ff81-fa32-4381-94c5-9d030e4464a3 None admin] PATH_INFO: `/v3/projects/39e472078e7d42f59243230e66c387f8/groups/6d7d63731f51414185728735a19d856f/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:57.465293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b6ff81-fa32-4381-94c5-9d030e4464a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=39e472078e7d42f59243230e66c387f8, group_id=6d7d63731f51414185728735a19d856f, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:57.467253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b6ff81-fa32-4381-94c5-9d030e4464a3 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:57.494594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 200/405] 10.222.0.121 () {58 vars in 1447 bytes} [Thu Aug 4 10:18:57 2022] PUT /identity/v3/projects/39e472078e7d42f59243230e66c387f8/groups/6d7d63731f51414185728735a19d856f/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:18:57.495863 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6ff81-fa32-4381-94c5-9d030e4464a3 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:57.545293 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 206/406] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:57 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:57.547035 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b5233-53bb-4147-b3f4-dd47f6ef5454 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:57.547580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b5233-53bb-4147-b3f4-dd47f6ef5454 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:57.562249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b5233-53bb-4147-b3f4-dd47f6ef5454 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:57.568036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 207/407] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:57 2022] GET /identity => generated 271 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:57.570191 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659a0f2-b217-47c6-b92a-7f17a030a17a None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:57.570487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659a0f2-b217-47c6-b92a-7f17a030a17a None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:57.570772 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659a0f2-b217-47c6-b92a-7f17a030a17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:57.608223 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-b659a0f2-b217-47c6-b92a-7f17a030a17a None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:57.610484 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-b659a0f2-b217-47c6-b92a-7f17a030a17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:57.664289 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6ff81-fa32-4381-94c5-9d030e4464a3 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:57.693976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 208/408] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:57 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:57.695072 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659a0f2-b217-47c6-b92a-7f17a030a17a None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:57.707455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d13cb-7fad-498c-8680-8809f701a76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:57.708490 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d13cb-7fad-498c-8680-8809f701a76a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:57.709510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d13cb-7fad-498c-8680-8809f701a76a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:57.710061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d13cb-7fad-498c-8680-8809f701a76a None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:57.710982 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d13cb-7fad-498c-8680-8809f701a76a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:57.711474 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d13cb-7fad-498c-8680-8809f701a76a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:57.713108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d13cb-7fad-498c-8680-8809f701a76a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:57.716233 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:57.724781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 201/409] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:18:57 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:57.726116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d13cb-7fad-498c-8680-8809f701a76a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:57.782622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:57.821040 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659a0f2-b217-47c6-b92a-7f17a030a17a None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:57.861203 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f9673a-1d53-465b-ab39-65e9b5c18346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:57.861989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9673a-1d53-465b-ab39-65e9b5c18346 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:57.862369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9673a-1d53-465b-ab39-65e9b5c18346 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:57.862780 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9673a-1d53-465b-ab39-65e9b5c18346 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:57.864919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d13cb-7fad-498c-8680-8809f701a76a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:57.881665 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f9673a-1d53-465b-ab39-65e9b5c18346 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:57.887645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-67f9673a-1d53-465b-ab39-65e9b5c18346 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:18:57.887645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:57.887645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:57.887645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:57.887645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:57.887645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:57.887645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:57.887645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:57.887645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:57.887645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:57.887645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:57.887645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:57.887645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:57.887645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:57.887645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:57.887645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:57.888795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:57.888795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:57.888795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:57.888795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:57.888795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:57.888795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:57.888795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:57.888795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:57.888795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:57.888795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:57.888795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:57.888795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:57.888795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:57.888795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:57.888795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:57.888795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:57.889645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:57.889645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:57.889645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:57.889645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:57.889645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:57.889645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:57.889645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:57.889645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:57.889645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:57.889645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:57.889645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:18:57.889645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:57.892025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 209/410] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:18:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:57.893168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f9673a-1d53-465b-ab39-65e9b5c18346 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:57.912472 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24f4f0-a560-40fb-a764-309d001b47fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:57.913671 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24f4f0-a560-40fb-a764-309d001b47fc None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:57.914196 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24f4f0-a560-40fb-a764-309d001b47fc None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:57.914736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24f4f0-a560-40fb-a764-309d001b47fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:57.927561 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24f4f0-a560-40fb-a764-309d001b47fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:57.933179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-cd24f4f0-a560-40fb-a764-309d001b47fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:57.933179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:57.933179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:57.933179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:57.933179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:57.933179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:57.933179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:57.933179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:57.933179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:57.933179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:57.933179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:57.933179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:57.933179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:57.933179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:57.933179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:57.933179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:57.934148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:57.934148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:57.934148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:57.934148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:57.934148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:57.934148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:57.934148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:57.934148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:57.934148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:57.934148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:57.934148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:57.934148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:57.934148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:57.934148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:57.934148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:57.934148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:57.934973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:57.934973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:57.934973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:57.934973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:57.934973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:57.934973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:57.934973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:57.934973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:57.934973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:57.934973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:57.934973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:18:57.934973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:18:57.937445 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 202/411] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:57.938482 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24f4f0-a560-40fb-a764-309d001b47fc None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:58.004901 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:58.034964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f9673a-1d53-465b-ab39-65e9b5c18346 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:58.051931 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24f4f0-a560-40fb-a764-309d001b47fc None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:58.076070 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:58.085437 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f08154-4a61-4b03-99d7-2df065c5ab9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:58.086677 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f08154-4a61-4b03-99d7-2df065c5ab9c None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:58.087119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f08154-4a61-4b03-99d7-2df065c5ab9c None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:58.087759 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f08154-4a61-4b03-99d7-2df065c5ab9c None admin] PATH_INFO: `/v3/groups/6d7d63731f51414185728735a19d856f` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:58.089958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f08154-4a61-4b03-99d7-2df065c5ab9c None admin] RBAC: Authorizing `identity:get_group(group_id=6d7d63731f51414185728735a19d856f)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:58.092669 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f08154-4a61-4b03-99d7-2df065c5ab9c None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:58.095574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 210/412] 10.222.0.121 () {56 vars in 1185 bytes} [Thu Aug 4 10:18:57 2022] GET /identity/v3/groups/6d7d63731f51414185728735a19d856f => generated 227 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:58.096649 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f08154-4a61-4b03-99d7-2df065c5ab9c None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:58.105805 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f8fd60-cc5b-439c-89e5-89ef809fb0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:58.106764 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8fd60-cc5b-439c-89e5-89ef809fb0fa None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:58.107202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8fd60-cc5b-439c-89e5-89ef809fb0fa None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:58.107627 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8fd60-cc5b-439c-89e5-89ef809fb0fa None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:58.109163 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f8fd60-cc5b-439c-89e5-89ef809fb0fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:58.109663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f8fd60-cc5b-439c-89e5-89ef809fb0fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:58.111638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f8fd60-cc5b-439c-89e5-89ef809fb0fa None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:58.124887 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 203/413] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:57 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:58.126075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f8fd60-cc5b-439c-89e5-89ef809fb0fa None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:58.220045 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f08154-4a61-4b03-99d7-2df065c5ab9c None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:58.237306 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f8fd60-cc5b-439c-89e5-89ef809fb0fa None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:58.260044 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f40b6-7d60-43d6-872c-77dc30aec9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:58.261294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:58.261878 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f40b6-7d60-43d6-872c-77dc30aec9bf None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:58.262174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f40b6-7d60-43d6-872c-77dc30aec9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:58.262455 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f40b6-7d60-43d6-872c-77dc30aec9bf None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:58.263240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f40b6-7d60-43d6-872c-77dc30aec9bf None admin] RBAC: Adding query filter params (name=member) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:58.263570 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f40b6-7d60-43d6-872c-77dc30aec9bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:58.265197 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f40b6-7d60-43d6-872c-77dc30aec9bf None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:58.281339 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 211/414] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:18:58 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:58.281339 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f40b6-7d60-43d6-872c-77dc30aec9bf None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:58.285855 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85064e-6b45-4afb-a154-9dc5e466717a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:58.286772 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85064e-6b45-4afb-a154-9dc5e466717a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:58.287216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85064e-6b45-4afb-a154-9dc5e466717a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:58.287683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85064e-6b45-4afb-a154-9dc5e466717a None admin] PATH_INFO: `/v3/projects/39e472078e7d42f59243230e66c387f8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:58.289132 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec85064e-6b45-4afb-a154-9dc5e466717a None admin] RBAC: Authorizing `identity:get_project(project_id=39e472078e7d42f59243230e66c387f8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:58.290773 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec85064e-6b45-4afb-a154-9dc5e466717a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:58.292732 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 204/415] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:58 2022] GET /identity/v3/projects/39e472078e7d42f59243230e66c387f8 => generated 303 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:58.293665 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85064e-6b45-4afb-a154-9dc5e466717a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:58.353218 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:58.404032 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f40b6-7d60-43d6-872c-77dc30aec9bf None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:58.409720 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85064e-6b45-4afb-a154-9dc5e466717a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:58.441315 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f20eef7-3fd8-4f31-81c2-dafc93f0b1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:58.441982 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20eef7-3fd8-4f31-81c2-dafc93f0b1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:58.442267 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20eef7-3fd8-4f31-81c2-dafc93f0b1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:58.442544 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20eef7-3fd8-4f31-81c2-dafc93f0b1c0 None admin] PATH_INFO: `/v3/groups/9c252f85b3564ef3a9bc3ac49cd07405` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:58.444062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f20eef7-3fd8-4f31-81c2-dafc93f0b1c0 None admin] RBAC: Authorizing `identity:get_group(group_id=9c252f85b3564ef3a9bc3ac49cd07405)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:58.445955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f20eef7-3fd8-4f31-81c2-dafc93f0b1c0 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:58.446736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e174c34-2a26-477d-a9dd-4aadc8acd0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:58.447825 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 212/416] 10.222.0.121 () {56 vars in 1185 bytes} [Thu Aug 4 10:18:58 2022] GET /identity/v3/groups/9c252f85b3564ef3a9bc3ac49cd07405 => generated 230 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:58.448090 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e174c34-2a26-477d-a9dd-4aadc8acd0dd None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:58.448589 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e174c34-2a26-477d-a9dd-4aadc8acd0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:58.449189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e174c34-2a26-477d-a9dd-4aadc8acd0dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:58.452681 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f20eef7-3fd8-4f31-81c2-dafc93f0b1c0 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:58.527427 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:58.546084 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e174c34-2a26-477d-a9dd-4aadc8acd0dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:58.547576 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e174c34-2a26-477d-a9dd-4aadc8acd0dd None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:58.577550 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f20eef7-3fd8-4f31-81c2-dafc93f0b1c0 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:58.617083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:58.629746 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98598d43-7f7b-4b5a-a9c0-095971d60570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:58.631457 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98598d43-7f7b-4b5a-a9c0-095971d60570 None admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:58.632214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98598d43-7f7b-4b5a-a9c0-095971d60570 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:58.632716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98598d43-7f7b-4b5a-a9c0-095971d60570 None admin] PATH_INFO: `/v3/projects/39e472078e7d42f59243230e66c387f8/groups/6d7d63731f51414185728735a19d856f/roles/89b847fca3574b9083287279aee0f773` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:58.636285 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98598d43-7f7b-4b5a-a9c0-095971d60570 None admin] RBAC: Authorizing `identity:create_grant(project_id=39e472078e7d42f59243230e66c387f8, group_id=6d7d63731f51414185728735a19d856f, role_id=89b847fca3574b9083287279aee0f773)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:58.638260 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98598d43-7f7b-4b5a-a9c0-095971d60570 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:58.668287 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 213/417] 10.222.0.121 () {58 vars in 1447 bytes} [Thu Aug 4 10:18:58 2022] PUT /identity/v3/projects/39e472078e7d42f59243230e66c387f8/groups/6d7d63731f51414185728735a19d856f/roles/89b847fca3574b9083287279aee0f773 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:18:58.671198 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98598d43-7f7b-4b5a-a9c0-095971d60570 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:58.695643 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 205/418] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:18:58 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:18:58.703660 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e174c34-2a26-477d-a9dd-4aadc8acd0dd None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:58.899329 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:58.906563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98598d43-7f7b-4b5a-a9c0-095971d60570 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:58.918697 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e174c34-2a26-477d-a9dd-4aadc8acd0dd None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:58.953174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdcd58c-9616-4a2d-98ef-3d34df4d2e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:58.954281 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcd58c-9616-4a2d-98ef-3d34df4d2e93 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:58.954848 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcd58c-9616-4a2d-98ef-3d34df4d2e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:58.955329 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcd58c-9616-4a2d-98ef-3d34df4d2e93 None admin] PATH_INFO: `/v3/projects/03f48ab98257418c836fe2e290bdf749` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:58.957006 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdcd58c-9616-4a2d-98ef-3d34df4d2e93 None admin] RBAC: Authorizing `identity:get_project(project_id=03f48ab98257418c836fe2e290bdf749)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:58.958753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdcd58c-9616-4a2d-98ef-3d34df4d2e93 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:58.961388 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 214/419] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:58 2022] GET /identity/v3/projects/03f48ab98257418c836fe2e290bdf749 => generated 345 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:58.967137 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2acc48e-85a7-448c-b32f-7cda644fdf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:58.968200 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2acc48e-85a7-448c-b32f-7cda644fdf85 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:58.968452 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdcd58c-9616-4a2d-98ef-3d34df4d2e93 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:58.968867 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2acc48e-85a7-448c-b32f-7cda644fdf85 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:58.969378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2acc48e-85a7-448c-b32f-7cda644fdf85 None admin] PATH_INFO: `/v3/projects/6d7a7a0b84384f14849056212228badb` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:58.970773 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2acc48e-85a7-448c-b32f-7cda644fdf85 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7a7a0b84384f14849056212228badb)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:58.972945 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2acc48e-85a7-448c-b32f-7cda644fdf85 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:58.976532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 206/420] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:18:58 2022] GET /identity/v3/projects/6d7a7a0b84384f14849056212228badb => generated 299 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:18:58.982824 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2acc48e-85a7-448c-b32f-7cda644fdf85 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:59.084695 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:59.176458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdcd58c-9616-4a2d-98ef-3d34df4d2e93 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:59.177853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2acc48e-85a7-448c-b32f-7cda644fdf85 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:59.219636 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020dfdc-1bbc-4d7b-a476-45f47f07ec01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:59.220857 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020dfdc-1bbc-4d7b-a476-45f47f07ec01 None admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:59.221328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020dfdc-1bbc-4d7b-a476-45f47f07ec01 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:59.221786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020dfdc-1bbc-4d7b-a476-45f47f07ec01 None admin] PATH_INFO: `/v3/projects/03f48ab98257418c836fe2e290bdf749/groups/9c252f85b3564ef3a9bc3ac49cd07405/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:59.225298 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020dfdc-1bbc-4d7b-a476-45f47f07ec01 None admin] RBAC: Authorizing `identity:create_grant(project_id=03f48ab98257418c836fe2e290bdf749, group_id=9c252f85b3564ef3a9bc3ac49cd07405, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:59.227228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020dfdc-1bbc-4d7b-a476-45f47f07ec01 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:59.254632 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03a071-b067-4705-8520-34063cf549ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:59.257607 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 215/421] 10.222.0.121 () {58 vars in 1447 bytes} [Thu Aug 4 10:18:58 2022] PUT /identity/v3/projects/03f48ab98257418c836fe2e290bdf749/groups/9c252f85b3564ef3a9bc3ac49cd07405/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:18:59.257995 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03a071-b067-4705-8520-34063cf549ca None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:59.258549 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03a071-b067-4705-8520-34063cf549ca None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:59.259094 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03a071-b067-4705-8520-34063cf549ca None admin] PATH_INFO: `/v3/groups/6d7d63731f51414185728735a19d856f` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:59.260878 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03a071-b067-4705-8520-34063cf549ca None admin] RBAC: Authorizing `identity:get_group(group_id=6d7d63731f51414185728735a19d856f)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:59.262970 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03a071-b067-4705-8520-34063cf549ca None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:59.265807 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 207/422] 10.222.0.121 () {56 vars in 1185 bytes} [Thu Aug 4 10:18:58 2022] GET /identity/v3/groups/6d7d63731f51414185728735a19d856f => generated 227 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:59.272744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020dfdc-1bbc-4d7b-a476-45f47f07ec01 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:59.390215 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66447970-a5a8-4410-a514-d8b6db7870e3 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:59.390914 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66447970-a5a8-4410-a514-d8b6db7870e3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:59.391421 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66447970-a5a8-4410-a514-d8b6db7870e3 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:59.393154 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 208/423] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:59 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:59.412089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19c6db-8b98-4f8e-a434-a9669aa31cbc None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:59.412411 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19c6db-8b98-4f8e-a434-a9669aa31cbc None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:59.412923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19c6db-8b98-4f8e-a434-a9669aa31cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:59.447341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-af19c6db-8b98-4f8e-a434-a9669aa31cbc None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:59.448741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-af19c6db-8b98-4f8e-a434-a9669aa31cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:59.456213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020dfdc-1bbc-4d7b-a476-45f47f07ec01 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:59.481029 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:59.487866 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:59.514434 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af195a9-3100-4f90-b203-0f1018f96dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:59.515141 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af195a9-3100-4f90-b203-0f1018f96dff None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:59.515427 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af195a9-3100-4f90-b203-0f1018f96dff None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:59.515706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af195a9-3100-4f90-b203-0f1018f96dff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:59.516581 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af195a9-3100-4f90-b203-0f1018f96dff None admin] RBAC: Adding query filter params (group.id=6d7d63731f51414185728735a19d856f, role.id=1b988376d12749bd8d653baa89493c5c, scope.project.id=6d7a7a0b84384f14849056212228badb) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:18:59.516903 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af195a9-3100-4f90-b203-0f1018f96dff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:18:59.518452 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af195a9-3100-4f90-b203-0f1018f96dff None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:18:59.525400 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 216/424] 10.222.0.121 () {56 vars in 1381 bytes} [Thu Aug 4 10:18:59 2022] GET /identity/v3/role_assignments?group.id=6d7d63731f51414185728735a19d856f&scope.project.id=6d7a7a0b84384f14849056212228badb&role.id=1b988376d12749bd8d653baa89493c5c => generated 613 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:18:59.555033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 209/425] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:59 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:59.581713 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af195a9-3100-4f90-b203-0f1018f96dff None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:59.705294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af195a9-3100-4f90-b203-0f1018f96dff None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:59.746045 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa73dd-1ad5-45da-bfe0-76746fc40d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:18:59.746716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa73dd-1ad5-45da-bfe0-76746fc40d24 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:59.747062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa73dd-1ad5-45da-bfe0-76746fc40d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:59.747347 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa73dd-1ad5-45da-bfe0-76746fc40d24 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:59.760869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa73dd-1ad5-45da-bfe0-76746fc40d24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:18:59.767025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-03fa73dd-1ad5-45da-bfe0-76746fc40d24 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:18:59.767025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:18:59.767025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:18:59.767025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:18:59.767025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:18:59.767025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:18:59.767025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:18:59.767025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:18:59.767025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:18:59.767025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:18:59.767025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:18:59.767025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:18:59.767025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:18:59.767025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:18:59.767025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:18:59.767025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:18:59.768055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:18:59.768055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:18:59.768055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:18:59.768055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:18:59.768055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:18:59.768055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:18:59.768055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:18:59.768055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:18:59.768055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:18:59.768055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:18:59.768055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:18:59.768055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:18:59.768055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:18:59.768055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:18:59.768055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:18:59.768055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:18:59.768952 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:18:59.768952 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:18:59.768952 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:18:59.768952 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:18:59.768952 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:18:59.768952 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:18:59.768952 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:18:59.768952 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:18:59.768952 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:18:59.768952 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:18:59.768952 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:18:59.768952 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:18:59.768952 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 217/426] 10.222.0.121 () {56 vars in 1119 bytes} [Thu Aug 4 10:18:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:18:59.775769 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19c6db-8b98-4f8e-a434-a9669aa31cbc None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:18:59.796458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77c864-cda0-4dd9-b3dd-91d6d434eb3e None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:59.796833 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77c864-cda0-4dd9-b3dd-91d6d434eb3e None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:59.796994 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77c864-cda0-4dd9-b3dd-91d6d434eb3e None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:59.798913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 218/427] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:18:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:18:59.811897 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69422f0-3aa1-4855-b52a-6f98f78e4d1c None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:18:59.812328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69422f0-3aa1-4855-b52a-6f98f78e4d1c None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:18:59.812569 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69422f0-3aa1-4855-b52a-6f98f78e4d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:18:59.842589 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-b69422f0-3aa1-4855-b52a-6f98f78e4d1c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:18:59.843973 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-b69422f0-3aa1-4855-b52a-6f98f78e4d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:18:59.880108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:18:59.928113 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19c6db-8b98-4f8e-a434-a9669aa31cbc None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:18:59.934173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 219/428] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:18:59 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:18:59.964942 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69422f0-3aa1-4855-b52a-6f98f78e4d1c None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:00.002395 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133f7c5-1bb8-426f-9713-ae489c49b13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:00.003070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133f7c5-1bb8-426f-9713-ae489c49b13a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:00.003380 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133f7c5-1bb8-426f-9713-ae489c49b13a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:00.004007 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133f7c5-1bb8-426f-9713-ae489c49b13a None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:00.004771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133f7c5-1bb8-426f-9713-ae489c49b13a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:00.005083 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133f7c5-1bb8-426f-9713-ae489c49b13a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:00.007892 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133f7c5-1bb8-426f-9713-ae489c49b13a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:00.030129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 210/429] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:18:59 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:00.038076 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133f7c5-1bb8-426f-9713-ae489c49b13a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:00.088775 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:00.123697 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69422f0-3aa1-4855-b52a-6f98f78e4d1c None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:00.169903 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e079930-2a9e-44ed-922d-2cca35a82211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:00.171096 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e079930-2a9e-44ed-922d-2cca35a82211 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:00.171446 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e079930-2a9e-44ed-922d-2cca35a82211 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:00.171793 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e079930-2a9e-44ed-922d-2cca35a82211 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:00.190668 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e079930-2a9e-44ed-922d-2cca35a82211 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:00.202808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-1e079930-2a9e-44ed-922d-2cca35a82211 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:19:00.202808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:00.202808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:00.202808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:00.202808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:00.202808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:00.202808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:00.202808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:00.202808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:00.202808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:00.202808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:00.202808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:00.202808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:00.202808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:00.202808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:00.202808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:00.203883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:00.203883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:00.203883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:00.203883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:00.203883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:00.203883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:00.203883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:00.203883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:00.203883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:00.203883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:00.203883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:00.203883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:00.203883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:00.203883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:00.203883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:00.203883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:00.205101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:00.205101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:00.205101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:00.205101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:00.205101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:00.205101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:00.205101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:00.205101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:00.205101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:00.205101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:00.205101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 10:19:00.205101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:00.207683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 220/430] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:18:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:00.217434 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e079930-2a9e-44ed-922d-2cca35a82211 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:00.221823 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133f7c5-1bb8-426f-9713-ae489c49b13a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:00.273347 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be334b15-64f1-4dc3-b3c2-4e76114e43ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:00.275316 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be334b15-64f1-4dc3-b3c2-4e76114e43ec None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:00.276263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be334b15-64f1-4dc3-b3c2-4e76114e43ec None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:00.277137 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be334b15-64f1-4dc3-b3c2-4e76114e43ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:00.281026 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:00.399438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e079930-2a9e-44ed-922d-2cca35a82211 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:00.413010 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be334b15-64f1-4dc3-b3c2-4e76114e43ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:00.414626 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be334b15-64f1-4dc3-b3c2-4e76114e43ec None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:00.437281 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:00.443985 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fca9e0-f75c-4224-8d76-477b0e61fb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:00.445477 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fca9e0-f75c-4224-8d76-477b0e61fb5a None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:00.446004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fca9e0-f75c-4224-8d76-477b0e61fb5a None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:00.446479 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fca9e0-f75c-4224-8d76-477b0e61fb5a None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:00.447416 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fca9e0-f75c-4224-8d76-477b0e61fb5a None admin] RBAC: Adding query filter params (name=member) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:00.447925 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fca9e0-f75c-4224-8d76-477b0e61fb5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:00.449637 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fca9e0-f75c-4224-8d76-477b0e61fb5a None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:00.464048 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 221/431] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:19:00 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:00.466518 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fca9e0-f75c-4224-8d76-477b0e61fb5a None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:00.562411 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 211/432] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:19:00 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 10:19:00.569448 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be334b15-64f1-4dc3-b3c2-4e76114e43ec None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:00.590634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:00.605815 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fca9e0-f75c-4224-8d76-477b0e61fb5a None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:00.652783 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6ee51-ece3-449f-a9b7-da254bc211cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:00.653877 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6ee51-ece3-449f-a9b7-da254bc211cb None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:00.654393 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6ee51-ece3-449f-a9b7-da254bc211cb None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:00.654871 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6ee51-ece3-449f-a9b7-da254bc211cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:00.708960 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be334b15-64f1-4dc3-b3c2-4e76114e43ec None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:00.733392 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:00.739112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6ee51-ece3-449f-a9b7-da254bc211cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:00.741187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6ee51-ece3-449f-a9b7-da254bc211cb None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:00.756456 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12980906-cb99-4082-b003-aa5c8ebca8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:00.757147 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12980906-cb99-4082-b003-aa5c8ebca8c1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:00.757441 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12980906-cb99-4082-b003-aa5c8ebca8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:00.757720 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12980906-cb99-4082-b003-aa5c8ebca8c1 None admin] PATH_INFO: `/v3/projects/6d7a7a0b84384f14849056212228badb` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:00.759142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12980906-cb99-4082-b003-aa5c8ebca8c1 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7a7a0b84384f14849056212228badb)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:00.760673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12980906-cb99-4082-b003-aa5c8ebca8c1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:00.762574 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 212/433] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:19:00 2022] GET /identity/v3/projects/6d7a7a0b84384f14849056212228badb => generated 299 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:00.769320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12980906-cb99-4082-b003-aa5c8ebca8c1 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:00.894316 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12980906-cb99-4082-b003-aa5c8ebca8c1 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:00.900673 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:00.902789 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 222/434] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:19:00 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:19:00.904040 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6ee51-ece3-449f-a9b7-da254bc211cb None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:00.936986 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e90ba05-27af-4029-92d8-e5b5ea635f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:00.937676 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90ba05-27af-4029-92d8-e5b5ea635f5a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:00.937965 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90ba05-27af-4029-92d8-e5b5ea635f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:00.938238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90ba05-27af-4029-92d8-e5b5ea635f5a None admin] PATH_INFO: `/v3/groups/6d7d63731f51414185728735a19d856f` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:00.939825 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e90ba05-27af-4029-92d8-e5b5ea635f5a None admin] RBAC: Authorizing `identity:get_group(group_id=6d7d63731f51414185728735a19d856f)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:00.941470 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e90ba05-27af-4029-92d8-e5b5ea635f5a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:00.948101 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 213/435] 10.222.0.121 () {56 vars in 1185 bytes} [Thu Aug 4 10:19:00 2022] GET /identity/v3/groups/6d7d63731f51414185728735a19d856f => generated 227 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:00.949711 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e90ba05-27af-4029-92d8-e5b5ea635f5a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:01.012419 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:01.019400 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6ee51-ece3-449f-a9b7-da254bc211cb None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:01.057008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480cfd41-1918-4b98-8932-38cd270ec43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:01.057740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480cfd41-1918-4b98-8932-38cd270ec43a None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:01.058173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480cfd41-1918-4b98-8932-38cd270ec43a None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:01.058437 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480cfd41-1918-4b98-8932-38cd270ec43a None admin] PATH_INFO: `/v3/projects/39e472078e7d42f59243230e66c387f8` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:01.059927 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480cfd41-1918-4b98-8932-38cd270ec43a None admin] RBAC: Authorizing `identity:get_project(project_id=39e472078e7d42f59243230e66c387f8)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:01.061670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480cfd41-1918-4b98-8932-38cd270ec43a None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:01.064439 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e90ba05-27af-4029-92d8-e5b5ea635f5a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:01.068765 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 223/436] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:19:00 2022] GET /identity/v3/projects/39e472078e7d42f59243230e66c387f8 => generated 303 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:01.069811 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480cfd41-1918-4b98-8932-38cd270ec43a None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:01.102699 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8c38b-f41c-4818-a24c-1d7831997aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:01.103819 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8c38b-f41c-4818-a24c-1d7831997aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:01.104433 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8c38b-f41c-4818-a24c-1d7831997aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:01.104961 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8c38b-f41c-4818-a24c-1d7831997aa8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:01.105984 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8c38b-f41c-4818-a24c-1d7831997aa8 None admin] RBAC: Adding query filter params (group.id=6d7d63731f51414185728735a19d856f, role.id=89b847fca3574b9083287279aee0f773, scope.project.id=6d7a7a0b84384f14849056212228badb) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:01.106537 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8c38b-f41c-4818-a24c-1d7831997aa8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:01.108321 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8c38b-f41c-4818-a24c-1d7831997aa8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:01.124525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 214/437] 10.222.0.121 () {56 vars in 1381 bytes} [Thu Aug 4 10:19:00 2022] GET /identity/v3/role_assignments?group.id=6d7d63731f51414185728735a19d856f&scope.project.id=6d7a7a0b84384f14849056212228badb&role.id=89b847fca3574b9083287279aee0f773 => generated 613 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:19:01.148583 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:01.192587 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480cfd41-1918-4b98-8932-38cd270ec43a None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:01.221631 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd7900-b136-4f5c-a5af-249bd174e586 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:01.221919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd7900-b136-4f5c-a5af-249bd174e586 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:01.222475 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd7900-b136-4f5c-a5af-249bd174e586 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:01.224007 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 215/438] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:01.230435 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81f0c2-e22f-498a-900a-38e627ee6c1d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:01.231141 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81f0c2-e22f-498a-900a-38e627ee6c1d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:01.231608 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81f0c2-e22f-498a-900a-38e627ee6c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:01.236623 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc931935-278c-4a9f-b605-ea9f25ad98fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:01.237320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc931935-278c-4a9f-b605-ea9f25ad98fc None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:01.237689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc931935-278c-4a9f-b605-ea9f25ad98fc None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:01.237975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc931935-278c-4a9f-b605-ea9f25ad98fc None admin] PATH_INFO: `/v3/groups/6d7d63731f51414185728735a19d856f` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:01.239675 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc931935-278c-4a9f-b605-ea9f25ad98fc None admin] RBAC: Authorizing `identity:get_group(group_id=6d7d63731f51414185728735a19d856f)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:01.241380 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc931935-278c-4a9f-b605-ea9f25ad98fc None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:01.243453 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 224/439] 10.222.0.121 () {56 vars in 1185 bytes} [Thu Aug 4 10:19:01 2022] GET /identity/v3/groups/6d7d63731f51414185728735a19d856f => generated 227 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:01.250192 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc931935-278c-4a9f-b605-ea9f25ad98fc None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:01.256830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-3a81f0c2-e22f-498a-900a-38e627ee6c1d None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:01.258552 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3a81f0c2-e22f-498a-900a-38e627ee6c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:01.359597 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 216/440] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:01 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:19:01.384460 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81f0c2-e22f-498a-900a-38e627ee6c1d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:01.401816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc931935-278c-4a9f-b605-ea9f25ad98fc None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:01.432651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:01.446811 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:01.449206 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0eb927-bc00-4dd6-8055-2b734617d9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:01.449759 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0eb927-bc00-4dd6-8055-2b734617d9a5 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:01.450126 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0eb927-bc00-4dd6-8055-2b734617d9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:01.450278 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0eb927-bc00-4dd6-8055-2b734617d9a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:01.451166 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0eb927-bc00-4dd6-8055-2b734617d9a5 None admin] RBAC: Adding query filter params (group.id=6d7d63731f51414185728735a19d856f, role.id=1b988376d12749bd8d653baa89493c5c, scope.project.id=39e472078e7d42f59243230e66c387f8) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:01.451650 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0eb927-bc00-4dd6-8055-2b734617d9a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:01.453875 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0eb927-bc00-4dd6-8055-2b734617d9a5 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:01.467997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 225/441] 10.222.0.121 () {56 vars in 1381 bytes} [Thu Aug 4 10:19:01 2022] GET /identity/v3/role_assignments?group.id=6d7d63731f51414185728735a19d856f&scope.project.id=39e472078e7d42f59243230e66c387f8&role.id=1b988376d12749bd8d653baa89493c5c => generated 613 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:19:01.522546 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81f0c2-e22f-498a-900a-38e627ee6c1d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:01.571315 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115f26c-ea37-45bf-b728-f0c5db6e9db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:01.572033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115f26c-ea37-45bf-b728-f0c5db6e9db3 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:01.572320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115f26c-ea37-45bf-b728-f0c5db6e9db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:01.572596 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115f26c-ea37-45bf-b728-f0c5db6e9db3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:01.590468 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115f26c-ea37-45bf-b728-f0c5db6e9db3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:01.593976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-8115f26c-ea37-45bf-b728-f0c5db6e9db3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:19:01.593976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:01.593976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:01.593976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:01.593976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:01.593976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:01.593976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:01.593976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:01.593976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:01.593976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:01.593976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:01.593976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:01.593976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:01.593976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:01.593976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:01.593976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:01.594967 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:01.594967 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:01.594967 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:01.594967 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:01.594967 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:01.594967 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:01.594967 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:01.594967 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:01.594967 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:01.594967 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:01.594967 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:01.594967 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:01.594967 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:01.594967 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:01.594967 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:01.594967 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:01.596249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:01.596249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:01.596249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:01.596249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:01.596249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:01.596249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:01.596249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:01.596249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:01.596249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:01.596249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:01.596249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:19:01.596249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:01.596249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 217/442] 10.222.0.121 () {56 vars in 1119 bytes} [Thu Aug 4 10:19:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:01.601915 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0eb927-bc00-4dd6-8055-2b734617d9a5 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:01.732479 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0eb927-bc00-4dd6-8055-2b734617d9a5 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:01.773063 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600148e1-0ad4-4f2b-8186-b7caf1020221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:01.773890 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600148e1-0ad4-4f2b-8186-b7caf1020221 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:01.774871 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600148e1-0ad4-4f2b-8186-b7caf1020221 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:01.774871 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600148e1-0ad4-4f2b-8186-b7caf1020221 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:01.775484 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600148e1-0ad4-4f2b-8186-b7caf1020221 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:01.775903 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600148e1-0ad4-4f2b-8186-b7caf1020221 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:01.777920 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600148e1-0ad4-4f2b-8186-b7caf1020221 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:01.787756 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 226/443] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:01 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:01.793827 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115f26c-ea37-45bf-b728-f0c5db6e9db3 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:01.836717 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176b2f3-76a4-4eb5-aa9e-399f38d8c450 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:01.836998 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176b2f3-76a4-4eb5-aa9e-399f38d8c450 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:01.837279 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176b2f3-76a4-4eb5-aa9e-399f38d8c450 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:01.838672 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 227/444] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 10:19:01.844690 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c021a06-f803-42a2-8c13-2436864444ef None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:01.845067 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c021a06-f803-42a2-8c13-2436864444ef None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:01.845426 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c021a06-f803-42a2-8c13-2436864444ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:01.869140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-9c021a06-f803-42a2-8c13-2436864444ef None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:01.870676 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-9c021a06-f803-42a2-8c13-2436864444ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:01.928130 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115f26c-ea37-45bf-b728-f0c5db6e9db3 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:01.937475 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:01.943532 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:01.965187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 228/445] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:01 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:19:01.968117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9ed0d-fd0b-42c1-b5a5-a4f03f0fbda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:01.968743 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9ed0d-fd0b-42c1-b5a5-a4f03f0fbda3 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:01.969030 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9ed0d-fd0b-42c1-b5a5-a4f03f0fbda3 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:01.969309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9ed0d-fd0b-42c1-b5a5-a4f03f0fbda3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:01.991352 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c021a06-f803-42a2-8c13-2436864444ef None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:02.046413 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9ed0d-fd0b-42c1-b5a5-a4f03f0fbda3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:02.047937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9ed0d-fd0b-42c1-b5a5-a4f03f0fbda3 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:02.091914 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c021a06-f803-42a2-8c13-2436864444ef None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:02.130544 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8eb98-1b6d-4bee-99b4-d34494cf9504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:02.131216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8eb98-1b6d-4bee-99b4-d34494cf9504 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:02.131581 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8eb98-1b6d-4bee-99b4-d34494cf9504 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:02.131859 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8eb98-1b6d-4bee-99b4-d34494cf9504 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:02.142796 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8eb98-1b6d-4bee-99b4-d34494cf9504 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:02.147798 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-48f8eb98-1b6d-4bee-99b4-d34494cf9504 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:19:02.147798 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:02.147798 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:02.147798 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:02.147798 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:02.147798 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:02.147798 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:02.147798 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:02.147798 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:02.147798 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:02.147798 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:02.147798 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:02.147798 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:02.147798 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:02.147798 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:02.147798 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:02.148788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:02.148788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:02.148788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:02.148788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:02.148788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:02.148788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:02.148788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:02.148788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:02.148788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:02.148788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:02.148788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:02.148788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:02.148788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:02.148788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:02.148788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:02.148788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:02.149616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:02.149616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:02.149616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:02.149616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:02.149616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:02.149616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:02.149616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:02.149616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:02.149616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:02.149616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:02.149616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:19:02.149616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:02.149616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 229/446] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:19:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:02.153948 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 218/447] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:19:01 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:19:02.155345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8eb98-1b6d-4bee-99b4-d34494cf9504 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:02.159659 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9ed0d-fd0b-42c1-b5a5-a4f03f0fbda3 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:02.167976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:02.202414 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:02.262480 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8eb98-1b6d-4bee-99b4-d34494cf9504 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:02.262889 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9ed0d-fd0b-42c1-b5a5-a4f03f0fbda3 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:02.298996 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c62462-fb80-45c0-b592-a050c36c1651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:02.299567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783a101-d3fb-44f8-a2f5-b0e96bc6e1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:02.299704 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c62462-fb80-45c0-b592-a050c36c1651 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:02.299918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c62462-fb80-45c0-b592-a050c36c1651 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:02.300277 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783a101-d3fb-44f8-a2f5-b0e96bc6e1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:02.300381 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c62462-fb80-45c0-b592-a050c36c1651 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:02.300576 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783a101-d3fb-44f8-a2f5-b0e96bc6e1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:02.300860 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783a101-d3fb-44f8-a2f5-b0e96bc6e1e6 None admin] PATH_INFO: `/v3/projects/39e472078e7d42f59243230e66c387f8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:02.301168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c62462-fb80-45c0-b592-a050c36c1651 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:02.301475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c62462-fb80-45c0-b592-a050c36c1651 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:02.302081 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783a101-d3fb-44f8-a2f5-b0e96bc6e1e6 None admin] RBAC: Authorizing `identity:get_project(project_id=39e472078e7d42f59243230e66c387f8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:02.302940 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c62462-fb80-45c0-b592-a050c36c1651 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:02.303594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783a101-d3fb-44f8-a2f5-b0e96bc6e1e6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:02.305358 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 219/448] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:19:02 2022] GET /identity/v3/projects/39e472078e7d42f59243230e66c387f8 => generated 303 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:19:02.311380 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783a101-d3fb-44f8-a2f5-b0e96bc6e1e6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:02.317586 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 230/449] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:19:02 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:19:02.327005 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c62462-fb80-45c0-b592-a050c36c1651 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:02.404919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:02.434865 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783a101-d3fb-44f8-a2f5-b0e96bc6e1e6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:02.444075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c62462-fb80-45c0-b592-a050c36c1651 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:02.446819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:02.472479 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382bfa95-5e3f-4a3b-a33c-957eb3e89472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:02.473251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bfa95-5e3f-4a3b-a33c-957eb3e89472 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:02.473436 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bfa95-5e3f-4a3b-a33c-957eb3e89472 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:02.473706 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bfa95-5e3f-4a3b-a33c-957eb3e89472 None admin] PATH_INFO: `/v3/groups/6d7d63731f51414185728735a19d856f` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:02.475016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382bfa95-5e3f-4a3b-a33c-957eb3e89472 None admin] RBAC: Authorizing `identity:get_group(group_id=6d7d63731f51414185728735a19d856f)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:02.476711 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382bfa95-5e3f-4a3b-a33c-957eb3e89472 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:02.478349 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 220/450] 10.222.0.121 () {56 vars in 1185 bytes} [Thu Aug 4 10:19:02 2022] GET /identity/v3/groups/6d7d63731f51414185728735a19d856f => generated 227 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:19:02.480250 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a3262-a0ee-40d7-a56b-7e56ef040877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:02.480936 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a3262-a0ee-40d7-a56b-7e56ef040877 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:02.481224 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a3262-a0ee-40d7-a56b-7e56ef040877 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:02.481534 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a3262-a0ee-40d7-a56b-7e56ef040877 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:02.484567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382bfa95-5e3f-4a3b-a33c-957eb3e89472 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:02.552170 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57a3262-a0ee-40d7-a56b-7e56ef040877 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:02.553629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57a3262-a0ee-40d7-a56b-7e56ef040877 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:02.587089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382bfa95-5e3f-4a3b-a33c-957eb3e89472 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:02.623301 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c6c5d-51dc-4274-9553-efff80e87def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:02.623925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c6c5d-51dc-4274-9553-efff80e87def None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:02.624206 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c6c5d-51dc-4274-9553-efff80e87def None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:02.624506 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c6c5d-51dc-4274-9553-efff80e87def None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:02.625277 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c6c5d-51dc-4274-9553-efff80e87def None admin] RBAC: Adding query filter params (group.id=6d7d63731f51414185728735a19d856f, role.id=89b847fca3574b9083287279aee0f773, scope.project.id=39e472078e7d42f59243230e66c387f8) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:02.625620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c6c5d-51dc-4274-9553-efff80e87def None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:02.627063 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c6c5d-51dc-4274-9553-efff80e87def None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:02.635788 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 221/451] 10.222.0.121 () {56 vars in 1381 bytes} [Thu Aug 4 10:19:02 2022] GET /identity/v3/role_assignments?group.id=6d7d63731f51414185728735a19d856f&scope.project.id=39e472078e7d42f59243230e66c387f8&role.id=89b847fca3574b9083287279aee0f773 => generated 613 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:02.666627 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 231/452] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:19:02 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:19:02.673478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c6c5d-51dc-4274-9553-efff80e87def None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:02.684351 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:02.798058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c6c5d-51dc-4274-9553-efff80e87def None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:02.839187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04141159-6b98-478f-8435-f2233dc979fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:02.839479 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04141159-6b98-478f-8435-f2233dc979fe None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:02.840379 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04141159-6b98-478f-8435-f2233dc979fe None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:02.840379 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04141159-6b98-478f-8435-f2233dc979fe None admin] PATH_INFO: `/v3/projects/03f48ab98257418c836fe2e290bdf749` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:02.841955 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04141159-6b98-478f-8435-f2233dc979fe None admin] RBAC: Authorizing `identity:get_project(project_id=03f48ab98257418c836fe2e290bdf749)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:02.844358 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04141159-6b98-478f-8435-f2233dc979fe None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:02.846549 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 222/453] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:19:02 2022] GET /identity/v3/projects/03f48ab98257418c836fe2e290bdf749 => generated 345 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:02.852638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a3262-a0ee-40d7-a56b-7e56ef040877 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:02.856178 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:02.982020 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a3262-a0ee-40d7-a56b-7e56ef040877 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:03.025054 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23401e4c-fc08-48d8-9626-f738525a0301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:03.025743 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23401e4c-fc08-48d8-9626-f738525a0301 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:03.026029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23401e4c-fc08-48d8-9626-f738525a0301 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:03.026307 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23401e4c-fc08-48d8-9626-f738525a0301 None admin] PATH_INFO: `/v3/groups/9c252f85b3564ef3a9bc3ac49cd07405` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:03.027575 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23401e4c-fc08-48d8-9626-f738525a0301 None admin] RBAC: Authorizing `identity:get_group(group_id=9c252f85b3564ef3a9bc3ac49cd07405)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:03.029095 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23401e4c-fc08-48d8-9626-f738525a0301 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:03.030828 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 232/454] 10.222.0.121 () {56 vars in 1185 bytes} [Thu Aug 4 10:19:02 2022] GET /identity/v3/groups/9c252f85b3564ef3a9bc3ac49cd07405 => generated 230 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:03.037325 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04141159-6b98-478f-8435-f2233dc979fe None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:03.124311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:03.152479 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04141159-6b98-478f-8435-f2233dc979fe None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:03.188208 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e0fb7-f69b-4b87-9654-c7e6f95d3fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:03.188843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e0fb7-f69b-4b87-9654-c7e6f95d3fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:03.189188 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e0fb7-f69b-4b87-9654-c7e6f95d3fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:03.189406 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e0fb7-f69b-4b87-9654-c7e6f95d3fc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:03.190190 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e0fb7-f69b-4b87-9654-c7e6f95d3fc1 None admin] RBAC: Adding query filter params (group.id=9c252f85b3564ef3a9bc3ac49cd07405, role.id=7342106a255b4bce8f150726602a8b95, scope.project.id=03f48ab98257418c836fe2e290bdf749) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:03.190510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e0fb7-f69b-4b87-9654-c7e6f95d3fc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:03.192004 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e0fb7-f69b-4b87-9654-c7e6f95d3fc1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:03.198771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 223/455] 10.222.0.121 () {56 vars in 1381 bytes} [Thu Aug 4 10:19:03 2022] GET /identity/v3/role_assignments?group.id=9c252f85b3564ef3a9bc3ac49cd07405&scope.project.id=03f48ab98257418c836fe2e290bdf749&role.id=7342106a255b4bce8f150726602a8b95 => generated 613 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:05.681525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0983b-e6dd-4d26-92b3-a539a18f3907 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:05.682605 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0983b-e6dd-4d26-92b3-a539a18f3907 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:05.683087 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0983b-e6dd-4d26-92b3-a539a18f3907 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:05.685081 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 233/456] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:05 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:05.691329 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896f73c-f50b-4c33-9c0c-6f5f38e2347b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:05.692345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896f73c-f50b-4c33-9c0c-6f5f38e2347b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:05.692826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896f73c-f50b-4c33-9c0c-6f5f38e2347b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:05.721070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a896f73c-f50b-4c33-9c0c-6f5f38e2347b None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:05.722774 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a896f73c-f50b-4c33-9c0c-6f5f38e2347b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:05.827142 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0199d-30a4-48cf-8b14-ac81f208db44 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:05.827917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0199d-30a4-48cf-8b14-ac81f208db44 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:05.828464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0199d-30a4-48cf-8b14-ac81f208db44 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:05.830172 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 234/457] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:05 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:05.836671 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d280e5-6972-4d27-a2b3-7721bbf58fd8 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:05.837306 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d280e5-6972-4d27-a2b3-7721bbf58fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:05.837436 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 224/458] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:05 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:19:05.837920 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d280e5-6972-4d27-a2b3-7721bbf58fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:05.863632 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896f73c-f50b-4c33-9c0c-6f5f38e2347b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:05.876824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-c5d280e5-6972-4d27-a2b3-7721bbf58fd8 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:05.878058 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c5d280e5-6972-4d27-a2b3-7721bbf58fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:06.002092 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:06.044587 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 235/459] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:05 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:19:06.056677 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896f73c-f50b-4c33-9c0c-6f5f38e2347b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:06.090410 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d280e5-6972-4d27-a2b3-7721bbf58fd8 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:06.094108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:06.110729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce096e7-7605-4a6b-af1c-e7f227b3c1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:06.111408 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce096e7-7605-4a6b-af1c-e7f227b3c1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:06.111688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce096e7-7605-4a6b-af1c-e7f227b3c1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:06.111976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce096e7-7605-4a6b-af1c-e7f227b3c1d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:06.126260 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce096e7-7605-4a6b-af1c-e7f227b3c1d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:06.128309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce096e7-7605-4a6b-af1c-e7f227b3c1d8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:06.149203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-3ce096e7-7605-4a6b-af1c-e7f227b3c1d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:06.149203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:06.149203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:06.149203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:06.149203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:06.149203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:06.149203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:06.149203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:06.149203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:06.149203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:06.149203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:06.149203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:06.149203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:06.149203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:06.149203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:06.150220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:06.150220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:06.150220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:06.150220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:06.150220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:06.150220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:06.150220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:06.150220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:06.150220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:06.150220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:06.150220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:06.150220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:06.150220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:06.150220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:06.150220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:06.150220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:06.150220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:06.151129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:06.151129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:06.151129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:06.151129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:06.151129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 225/460] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:06.156089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeba94e-c13a-4af4-b66b-91a46207ceda None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:06.156742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeba94e-c13a-4af4-b66b-91a46207ceda None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:06.157441 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeba94e-c13a-4af4-b66b-91a46207ceda None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:06.163622 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 226/461] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:06 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:06.164809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeba94e-c13a-4af4-b66b-91a46207ceda None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:06.215475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d280e5-6972-4d27-a2b3-7721bbf58fd8 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:06.253175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef7adf-e29b-41af-8c58-e26c695c6b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:06.254332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef7adf-e29b-41af-8c58-e26c695c6b19 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:06.254837 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef7adf-e29b-41af-8c58-e26c695c6b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:06.255332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef7adf-e29b-41af-8c58-e26c695c6b19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:06.267336 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef7adf-e29b-41af-8c58-e26c695c6b19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:06.269380 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef7adf-e29b-41af-8c58-e26c695c6b19 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:06.283368 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeba94e-c13a-4af4-b66b-91a46207ceda None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:06.286643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-1bef7adf-e29b-41af-8c58-e26c695c6b19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:06.286643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:06.286643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:06.286643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:06.286643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:06.286643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:06.286643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:06.286643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:06.286643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:06.286643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:06.286643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:06.286643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:06.286643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:06.286643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:06.286643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:06.287625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:06.287625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:06.287625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:06.287625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:06.287625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:06.287625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:06.287625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:06.287625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:06.287625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:06.287625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:06.287625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:06.287625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:06.287625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:06.287625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:06.287625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:06.287625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:06.287625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:06.288670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:06.288670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:06.288670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:06.288670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:06.291072 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 236/462] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:19:06.292778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4c9c4-97a6-496a-8197-462e055a8e10 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:06.293297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4c9c4-97a6-496a-8197-462e055a8e10 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:06.293773 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4c9c4-97a6-496a-8197-462e055a8e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:06.314495 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-4eb4c9c4-97a6-496a-8197-462e055a8e10 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:06.314960 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:06.316537 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-4eb4c9c4-97a6-496a-8197-462e055a8e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:06.332333 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bbb4fc-faec-46ef-bfa8-c9b9e9c1a7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:06.333333 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbb4fc-faec-46ef-bfa8-c9b9e9c1a7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:06.333333 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbb4fc-faec-46ef-bfa8-c9b9e9c1a7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:06.333639 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbb4fc-faec-46ef-bfa8-c9b9e9c1a7f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:06.334433 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bbb4fc-faec-46ef-bfa8-c9b9e9c1a7f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:06.334758 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bbb4fc-faec-46ef-bfa8-c9b9e9c1a7f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:06.336245 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bbb4fc-faec-46ef-bfa8-c9b9e9c1a7f7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:06.350654 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 227/463] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:06.355377 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bbb4fc-faec-46ef-bfa8-c9b9e9c1a7f7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:06.379166 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:06.399589 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 237/464] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:06 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:19:06.400873 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4c9c4-97a6-496a-8197-462e055a8e10 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:06.493472 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bbb4fc-faec-46ef-bfa8-c9b9e9c1a7f7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:06.504275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4c9c4-97a6-496a-8197-462e055a8e10 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:06.531846 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd8a31-9d15-4624-a93f-b5f06493fa4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:06.532537 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd8a31-9d15-4624-a93f-b5f06493fa4c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:06.532825 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd8a31-9d15-4624-a93f-b5f06493fa4c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:06.533105 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd8a31-9d15-4624-a93f-b5f06493fa4c None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:06.533937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd8a31-9d15-4624-a93f-b5f06493fa4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:06.534250 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd8a31-9d15-4624-a93f-b5f06493fa4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:06.535649 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd8a31-9d15-4624-a93f-b5f06493fa4c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:06.542298 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0b925-8548-410a-a099-3a3c54e46db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:06.543240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0b925-8548-410a-a099-3a3c54e46db6 None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:06.543726 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0b925-8548-410a-a099-3a3c54e46db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:06.544249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0b925-8548-410a-a099-3a3c54e46db6 None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:06.545151 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0b925-8548-410a-a099-3a3c54e46db6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:06.546854 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0b925-8548-410a-a099-3a3c54e46db6 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:06.552548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 228/465] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:06.553920 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd8a31-9d15-4624-a93f-b5f06493fa4c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:06.577539 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 238/466] 10.222.0.121 () {60 vars in 1137 bytes} [Thu Aug 4 10:19:06 2022] POST /identity/v3/users => generated 247 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:19:06.589124 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0b925-8548-410a-a099-3a3c54e46db6 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:06.605687 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:06.666160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:06.697757 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd8a31-9d15-4624-a93f-b5f06493fa4c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:06.754755 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9708a7bd-b9a9-4e38-91a8-7bb1ab22730e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:06.755981 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708a7bd-b9a9-4e38-91a8-7bb1ab22730e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:06.756585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708a7bd-b9a9-4e38-91a8-7bb1ab22730e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:06.757076 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708a7bd-b9a9-4e38-91a8-7bb1ab22730e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:06.771868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9708a7bd-b9a9-4e38-91a8-7bb1ab22730e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:06.773923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9708a7bd-b9a9-4e38-91a8-7bb1ab22730e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:06.793455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-9708a7bd-b9a9-4e38-91a8-7bb1ab22730e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:06.793455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:06.793455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:06.793455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:06.793455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:06.793455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:06.793455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:06.793455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:06.793455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:06.793455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:06.793455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:06.793455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:06.793455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:06.793455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:06.793455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:06.794452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:06.794452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:06.794452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:06.794452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:06.794452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:06.794452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:06.794452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:06.794452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:06.794452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:06.794452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:06.794452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:06.794452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:06.794452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:06.794452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:06.794452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:06.794452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:06.794452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:06.795374 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:06.795374 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:06.795374 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:06.795374 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:06.797492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 229/467] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:06.804533 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9708a7bd-b9a9-4e38-91a8-7bb1ab22730e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:06.824461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0b925-8548-410a-a099-3a3c54e46db6 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:06.878725 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c83aa-a43f-4e86-8b5e-3d92e8005783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:06.879420 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c83aa-a43f-4e86-8b5e-3d92e8005783 None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:06.879708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c83aa-a43f-4e86-8b5e-3d92e8005783 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:06.880016 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c83aa-a43f-4e86-8b5e-3d92e8005783 None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:06.880877 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48c83aa-a43f-4e86-8b5e-3d92e8005783 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:06.882540 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48c83aa-a43f-4e86-8b5e-3d92e8005783 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:06.911704 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 239/468] 10.222.0.121 () {60 vars in 1137 bytes} [Thu Aug 4 10:19:06 2022] POST /identity/v3/users => generated 249 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:19:06.913265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2ea67-a8b8-4a3c-89a6-160f3e8d5174 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:06.913572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2ea67-a8b8-4a3c-89a6-160f3e8d5174 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:06.913857 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2ea67-a8b8-4a3c-89a6-160f3e8d5174 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:06.915571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 240/469] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:06 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:06.924723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347f7cc-4139-4cd3-862e-329fc2b373e2 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:06.924991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347f7cc-4139-4cd3-862e-329fc2b373e2 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:06.925267 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347f7cc-4139-4cd3-862e-329fc2b373e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:06.953296 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-1347f7cc-4139-4cd3-862e-329fc2b373e2 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:06.954781 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-1347f7cc-4139-4cd3-862e-329fc2b373e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:06.965000 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:06.977988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9708a7bd-b9a9-4e38-91a8-7bb1ab22730e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:07.024209 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf73ae-492a-40bb-a557-edf3c1e50ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:07.025279 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf73ae-492a-40bb-a557-edf3c1e50ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:07.025748 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf73ae-492a-40bb-a557-edf3c1e50ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:07.026182 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf73ae-492a-40bb-a557-edf3c1e50ea5 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:07.027061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcf73ae-492a-40bb-a557-edf3c1e50ea5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:07.027526 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcf73ae-492a-40bb-a557-edf3c1e50ea5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:07.029215 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcf73ae-492a-40bb-a557-edf3c1e50ea5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:07.043703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 230/470] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:07.050289 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf73ae-492a-40bb-a557-edf3c1e50ea5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:07.063176 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 241/471] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:06 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:19:07.087979 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347f7cc-4139-4cd3-862e-329fc2b373e2 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:07.119674 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:07.176953 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf73ae-492a-40bb-a557-edf3c1e50ea5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:07.205005 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347f7cc-4139-4cd3-862e-329fc2b373e2 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:07.217443 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16907c74-8839-4053-9328-f839bc0b3f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:07.218569 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16907c74-8839-4053-9328-f839bc0b3f45 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:07.219094 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16907c74-8839-4053-9328-f839bc0b3f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:07.219553 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16907c74-8839-4053-9328-f839bc0b3f45 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:07.220922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16907c74-8839-4053-9328-f839bc0b3f45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:07.224019 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16907c74-8839-4053-9328-f839bc0b3f45 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:07.252235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e31ee1-10da-4663-8e66-a9caa91d1461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:07.253133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e31ee1-10da-4663-8e66-a9caa91d1461 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:07.253515 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e31ee1-10da-4663-8e66-a9caa91d1461 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:07.253927 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e31ee1-10da-4663-8e66-a9caa91d1461 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:07.263057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 231/472] 10.222.0.121 () {60 vars in 1137 bytes} [Thu Aug 4 10:19:07 2022] POST /identity/v3/users => generated 249 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:19:07.267843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e31ee1-10da-4663-8e66-a9caa91d1461 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:07.270126 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e31ee1-10da-4663-8e66-a9caa91d1461 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:07.298792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-63e31ee1-10da-4663-8e66-a9caa91d1461 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:07.298792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:07.298792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:07.298792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:07.298792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:07.298792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:07.298792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:07.298792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:07.298792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:07.298792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:07.298792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:07.298792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:07.298792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:07.298792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:07.298792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:07.300621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:07.300621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:07.300621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:07.300621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:07.300621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:07.300621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:07.300621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:07.300621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:07.300621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:07.300621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:07.300621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:07.300621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:07.300621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:07.300621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:07.300621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:07.300621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:07.300621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:07.302138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:07.302138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:07.302138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:07.302138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:07.302138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 242/473] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:19:07.315011 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16907c74-8839-4053-9328-f839bc0b3f45 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:07.332699 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:07.386530 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394a9b8-94bc-4f40-98ec-b169108eef32 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:07.388164 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394a9b8-94bc-4f40-98ec-b169108eef32 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:07.389156 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394a9b8-94bc-4f40-98ec-b169108eef32 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:07.391900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 243/474] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:07 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:07.400459 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896a715-9556-4251-b52e-3ecd8c3936c5 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:07.401310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896a715-9556-4251-b52e-3ecd8c3936c5 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:07.401804 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896a715-9556-4251-b52e-3ecd8c3936c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:07.428826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-1896a715-9556-4251-b52e-3ecd8c3936c5 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:07.430726 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-1896a715-9556-4251-b52e-3ecd8c3936c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:07.476971 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16907c74-8839-4053-9328-f839bc0b3f45 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:07.537182 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8ea81-f16f-4859-9056-9c04cd5f3628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:07.537921 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8ea81-f16f-4859-9056-9c04cd5f3628 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:07.538281 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8ea81-f16f-4859-9056-9c04cd5f3628 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:07.538562 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8ea81-f16f-4859-9056-9c04cd5f3628 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:07.539302 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8ea81-f16f-4859-9056-9c04cd5f3628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:07.539618 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8ea81-f16f-4859-9056-9c04cd5f3628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:07.540838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 244/475] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:07 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:19:07.543037 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8ea81-f16f-4859-9056-9c04cd5f3628 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:07.573669 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 232/476] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:07.575211 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896a715-9556-4251-b52e-3ecd8c3936c5 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:07.597676 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8ea81-f16f-4859-9056-9c04cd5f3628 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:07.602062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:07.733820 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:07.754041 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896a715-9556-4251-b52e-3ecd8c3936c5 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:07.786840 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8ea81-f16f-4859-9056-9c04cd5f3628 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:07.796871 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1c61b-8491-42db-8862-b1fde71f61f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:07.797593 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1c61b-8491-42db-8862-b1fde71f61f6 None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:07.797834 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1c61b-8491-42db-8862-b1fde71f61f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:07.798109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1c61b-8491-42db-8862-b1fde71f61f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:07.798881 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1c61b-8491-42db-8862-b1fde71f61f6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:07.800372 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1c61b-8491-42db-8862-b1fde71f61f6 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:07.814891 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.sql.core [None req-25c1c61b-8491-42db-8862-b1fde71f61f6 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 10:19:07.814891 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 04 10:19:07.814891 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [parameters: {'id': 'd12957139295492696a63fa00249a314', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 04 10:19:07.814891 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80235) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 10:19:07.832335 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b168a-5faa-4b3f-9f4d-399116d10d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:07.833275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-25c1c61b-8491-42db-8862-b1fde71f61f6 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 04 10:19:07.833275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:07.833275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 10:19:07.833275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 10:19:07.833275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 10:19:07.833275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 10:19:07.833275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 10:19:07.833275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 10:19:07.833275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 10:19:07.833275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application conn.query(q) Aug 04 10:19:07.833275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 10:19:07.833275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 10:19:07.833275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 10:19:07.833275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application result.read() Aug 04 10:19:07.833275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 10:19:07.834799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 10:19:07.834799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 10:19:07.834799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 10:19:07.834799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 10:19:07.834799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 10:19:07.834799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 10:19:07.834799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 10:19:07.834799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 10:19:07.834799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:07.834799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 10:19:07.834799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:07.834799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:07.834799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 10:19:07.834799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 10:19:07.834799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 04 10:19:07.834799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return ref.to_dict() Aug 04 10:19:07.834799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 10:19:07.834799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application next(self.gen) Aug 04 10:19:07.835919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 10:19:07.835919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application yield resource Aug 04 10:19:07.835919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 10:19:07.835919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application next(self.gen) Aug 04 10:19:07.835919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 10:19:07.835919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 10:19:07.835919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 10:19:07.835919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 10:19:07.835919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 10:19:07.835919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise self.value Aug 04 10:19:07.835919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 10:19:07.835919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 10:19:07.835919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 10:19:07.835919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application session.commit() Aug 04 10:19:07.835919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 10:19:07.835919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 10:19:07.835919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 10:19:07.836989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 10:19:07.836989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 10:19:07.836989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self.session.flush() Aug 04 10:19:07.836989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 10:19:07.836989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 10:19:07.836989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 10:19:07.836989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 10:19:07.836989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 10:19:07.836989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application compat.raise_( Aug 04 10:19:07.836989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 10:19:07.836989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception Aug 04 10:19:07.836989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 10:19:07.836989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 10:19:07.836989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 10:19:07.836989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 10:19:07.836989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 10:19:07.836989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 10:19:07.845241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 10:19:07.845241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 10:19:07.845241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 04 10:19:07.845241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 04 10:19:07.845241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 10:19:07.845241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 10:19:07.845241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 10:19:07.845241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 10:19:07.845241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 10:19:07.845241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 10:19:07.845241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 10:19:07.845241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 10:19:07.845241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 10:19:07.845241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 10:19:07.845241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 10:19:07.845241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception Aug 04 10:19:07.847565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 10:19:07.847565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 10:19:07.847565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 10:19:07.847565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 10:19:07.847565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 10:19:07.847565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 10:19:07.847565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 10:19:07.847565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application conn.query(q) Aug 04 10:19:07.847565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 10:19:07.847565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 10:19:07.847565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 10:19:07.847565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application result.read() Aug 04 10:19:07.847565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 10:19:07.847565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 10:19:07.847565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 10:19:07.847565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 10:19:07.850464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 10:19:07.850464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 10:19:07.850464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 10:19:07.850464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 10:19:07.850464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 10:19:07.850464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 04 10:19:07.850464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [parameters: {'id': 'd12957139295492696a63fa00249a314', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 04 10:19:07.850464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 10:19:07.850464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:07.850464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:19:07.850464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:07.850464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:07.850464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:07.850464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:07.850464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:07.850464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:07.853972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:07.853972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:07.853972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:07.853972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:07.853972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:07.853972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:07.853972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 04 10:19:07.853972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 04 10:19:07.853972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:07.853972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:07.853972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 04 10:19:07.853972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 04 10:19:07.853972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 10:19:07.853972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 10:19:07.853972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 04 10:19:07.853972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:07.857300 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 245/477] 10.222.0.121 () {60 vars in 1136 bytes} [Thu Aug 4 10:19:07 2022] POST /identity/v3/roles => generated 144 bytes in 263 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:19:07.857300 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1c61b-8491-42db-8862-b1fde71f61f6 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:07.857612 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b168a-5faa-4b3f-9f4d-399116d10d84 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:07.857612 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b168a-5faa-4b3f-9f4d-399116d10d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:07.857612 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b168a-5faa-4b3f-9f4d-399116d10d84 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:07.859023 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b168a-5faa-4b3f-9f4d-399116d10d84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:07.861266 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b168a-5faa-4b3f-9f4d-399116d10d84 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:07.935538 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 233/478] 10.222.0.121 () {60 vars in 1137 bytes} [Thu Aug 4 10:19:07 2022] POST /identity/v3/users => generated 250 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:19:07.980330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:08.041989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1c61b-8491-42db-8862-b1fde71f61f6 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:08.089987 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc4b0f-7c02-47fd-8608-a24a5f8e5220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:08.091157 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc4b0f-7c02-47fd-8608-a24a5f8e5220 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:08.091702 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc4b0f-7c02-47fd-8608-a24a5f8e5220 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:08.092233 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc4b0f-7c02-47fd-8608-a24a5f8e5220 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:08.106951 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc4b0f-7c02-47fd-8608-a24a5f8e5220 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:08.112577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-d0dc4b0f-7c02-47fd-8608-a24a5f8e5220 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:19:08.112577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:08.112577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:08.112577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:08.112577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:08.112577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:08.112577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:08.112577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:08.112577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:08.112577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:08.112577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:08.112577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:08.112577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:08.112577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:08.112577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:08.112577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:08.113953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:08.113953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:08.113953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:08.113953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:08.113953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:08.113953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:08.113953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:08.113953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:08.113953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:08.113953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:08.113953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:08.113953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:08.113953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:08.113953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:08.113953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:08.113953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:08.114802 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:08.114802 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:08.114802 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:08.114802 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:08.114802 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:08.114802 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:08.114802 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:08.114802 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:08.114802 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:08.114802 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:08.114802 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 10:19:08.114802 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:08.114802 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 246/479] 10.222.0.121 () {56 vars in 1119 bytes} [Thu Aug 4 10:19:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:08.122508 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b168a-5faa-4b3f-9f4d-399116d10d84 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:08.285701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b168a-5faa-4b3f-9f4d-399116d10d84 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:08.318046 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:08.330734 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1d77c-846f-4e7c-ab01-4730985f38c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:08.331889 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1d77c-846f-4e7c-ab01-4730985f38c1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:08.332634 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1d77c-846f-4e7c-ab01-4730985f38c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:08.332984 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1d77c-846f-4e7c-ab01-4730985f38c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:08.333994 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1d77c-846f-4e7c-ab01-4730985f38c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:08.334418 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1d77c-846f-4e7c-ab01-4730985f38c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:08.335908 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1d77c-846f-4e7c-ab01-4730985f38c1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:08.348222 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 234/480] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:08 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:09.648217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaac892-4928-4ab1-9726-935c1e467f2e None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:09.648830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaac892-4928-4ab1-9726-935c1e467f2e None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:09.648830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaac892-4928-4ab1-9726-935c1e467f2e None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:09.650589 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 247/481] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:09 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:09.656541 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab3ea0-2249-49dd-9dca-c5c79e342aff None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:09.657540 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab3ea0-2249-49dd-9dca-c5c79e342aff None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:09.657974 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab3ea0-2249-49dd-9dca-c5c79e342aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:09.668782 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca261de-743f-4d54-8a43-3ffc05e451da None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:09.669062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca261de-743f-4d54-8a43-3ffc05e451da None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:09.669347 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca261de-743f-4d54-8a43-3ffc05e451da None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:09.671132 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 248/482] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:09 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:09.678425 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ef3b4-9e7b-4c2f-961b-bd5efc8ba3b2 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:09.678709 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ef3b4-9e7b-4c2f-961b-bd5efc8ba3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:09.678991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ef3b4-9e7b-4c2f-961b-bd5efc8ba3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:09.687645 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-dfab3ea0-2249-49dd-9dca-c5c79e342aff None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:09.689025 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-dfab3ea0-2249-49dd-9dca-c5c79e342aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:09.715317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-ac1ef3b4-9e7b-4c2f-961b-bd5efc8ba3b2 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:09.716567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-ac1ef3b4-9e7b-4c2f-961b-bd5efc8ba3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:09.817170 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 249/483] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:09 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:19:09.849248 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ef3b4-9e7b-4c2f-961b-bd5efc8ba3b2 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:09.863935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 235/484] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:09 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:19:09.890984 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab3ea0-2249-49dd-9dca-c5c79e342aff None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:09.944340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:10.011512 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ef3b4-9e7b-4c2f-961b-bd5efc8ba3b2 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:10.048993 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d290798-21ee-4267-85d1-ec7b9141992d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:10.049667 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d290798-21ee-4267-85d1-ec7b9141992d None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:10.049955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d290798-21ee-4267-85d1-ec7b9141992d None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:10.050235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d290798-21ee-4267-85d1-ec7b9141992d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:10.063594 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d290798-21ee-4267-85d1-ec7b9141992d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:10.069075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-4d290798-21ee-4267-85d1-ec7b9141992d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:10.069075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:10.069075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:10.069075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:10.069075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:10.069075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:10.069075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:10.069075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:10.069075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:10.069075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:10.069075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:10.069075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:10.069075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:10.069075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:10.069075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:10.069075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:10.071622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:10.071622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:10.071622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:10.071622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:10.071622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:10.071622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:10.071622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:10.071622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:10.071622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:10.071622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:10.071622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:10.071622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:10.071622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:10.071622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:10.071622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:10.071622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:10.073470 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:10.073470 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:10.073470 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:10.073470 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:10.073470 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:10.073470 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:10.073470 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:10.073470 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:10.073470 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:10.073470 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:10.073470 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:10.073470 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:10.073470 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 250/485] 10.222.0.121 () {56 vars in 1107 bytes} [Thu Aug 4 10:19:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:10.074876 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab3ea0-2249-49dd-9dca-c5c79e342aff None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:10.075008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009dcfe-48dc-4e49-9c75-e636b07ebfdd None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:10.075008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009dcfe-48dc-4e49-9c75-e636b07ebfdd None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:10.075008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009dcfe-48dc-4e49-9c75-e636b07ebfdd None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:10.082291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 251/486] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:10 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:10.082291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009dcfe-48dc-4e49-9c75-e636b07ebfdd None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:10.094552 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:10.119503 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de5645-145b-402e-ba64-85fb3a397e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:10.120745 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de5645-145b-402e-ba64-85fb3a397e2c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:10.121026 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de5645-145b-402e-ba64-85fb3a397e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:10.121298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de5645-145b-402e-ba64-85fb3a397e2c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:10.135374 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de5645-145b-402e-ba64-85fb3a397e2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:10.141336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-02de5645-145b-402e-ba64-85fb3a397e2c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:10.141336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:10.141336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:10.141336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:10.141336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:10.141336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:10.141336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:10.141336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:10.141336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:10.141336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:10.141336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:10.141336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:10.141336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:10.141336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:10.141336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:10.141336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:10.142560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:10.142560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:10.142560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:10.142560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:10.142560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:10.142560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:10.142560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:10.142560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:10.142560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:10.142560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:10.142560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:10.142560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:10.142560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:10.142560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:10.142560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:10.142560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:10.143771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:10.143771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:10.143771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:10.143771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:10.143771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:10.143771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:10.143771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:10.143771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:10.143771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:10.143771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:10.143771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:10.143771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:10.149266 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 236/487] 10.222.0.121 () {56 vars in 1107 bytes} [Thu Aug 4 10:19:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:19:10.149974 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af406bab-f796-44a8-bb3f-fca4feb15321 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:10.150453 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af406bab-f796-44a8-bb3f-fca4feb15321 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:10.150735 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af406bab-f796-44a8-bb3f-fca4feb15321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:10.172907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-af406bab-f796-44a8-bb3f-fca4feb15321 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:10.174144 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-af406bab-f796-44a8-bb3f-fca4feb15321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:10.260242 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 237/488] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:10 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:19:10.260947 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af406bab-f796-44a8-bb3f-fca4feb15321 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:10.277973 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009dcfe-48dc-4e49-9c75-e636b07ebfdd None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:10.335824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:10.353551 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc332ec-119c-4a94-a701-fc24e0ef0cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:10.354500 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc332ec-119c-4a94-a701-fc24e0ef0cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:10.354970 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc332ec-119c-4a94-a701-fc24e0ef0cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:10.355420 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc332ec-119c-4a94-a701-fc24e0ef0cc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:10.356896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc332ec-119c-4a94-a701-fc24e0ef0cc3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:10.357375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc332ec-119c-4a94-a701-fc24e0ef0cc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:10.359551 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc332ec-119c-4a94-a701-fc24e0ef0cc3 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:10.372226 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 252/489] 10.222.0.121 () {56 vars in 1108 bytes} [Thu Aug 4 10:19:10 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:10.373411 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc332ec-119c-4a94-a701-fc24e0ef0cc3 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:10.407978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af406bab-f796-44a8-bb3f-fca4feb15321 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:10.421508 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:10.471324 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1603e96-2f36-43ef-b8de-d2eec439dc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:10.473081 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1603e96-2f36-43ef-b8de-d2eec439dc64 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:10.473081 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1603e96-2f36-43ef-b8de-d2eec439dc64 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:10.473412 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1603e96-2f36-43ef-b8de-d2eec439dc64 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:10.474757 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1603e96-2f36-43ef-b8de-d2eec439dc64 None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:10.475378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1603e96-2f36-43ef-b8de-d2eec439dc64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:10.478144 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1603e96-2f36-43ef-b8de-d2eec439dc64 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:10.495055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 238/490] 10.222.0.121 () {56 vars in 1108 bytes} [Thu Aug 4 10:19:10 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:10.500681 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1603e96-2f36-43ef-b8de-d2eec439dc64 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:10.506324 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc332ec-119c-4a94-a701-fc24e0ef0cc3 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:10.548254 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39cb31-aac3-423f-a3c2-e1dcdd55e932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:10.549373 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39cb31-aac3-423f-a3c2-e1dcdd55e932 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:10.549844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39cb31-aac3-423f-a3c2-e1dcdd55e932 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:10.550633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39cb31-aac3-423f-a3c2-e1dcdd55e932 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:10.577189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b39cb31-aac3-423f-a3c2-e1dcdd55e932 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:10.588263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-4b39cb31-aac3-423f-a3c2-e1dcdd55e932 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:10.588263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:10.588263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:10.588263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:10.588263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:10.588263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:10.588263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:10.588263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:10.588263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:10.588263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:10.588263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:10.588263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:10.588263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:10.588263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:10.588263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:10.588263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:10.589406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:10.589406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:10.589406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:10.589406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:10.589406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:10.589406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:10.589406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:10.589406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:10.589406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:10.589406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:10.589406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:10.589406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:10.589406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:10.589406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:10.589406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:10.589406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:10.590337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:10.590337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:10.590337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:10.590337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:10.590337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:10.590337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:10.590337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:10.590337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:10.590337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:10.590337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:10.590337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:10.590337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:10.591114 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 253/491] 10.222.0.121 () {56 vars in 1107 bytes} [Thu Aug 4 10:19:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:10.592660 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39cb31-aac3-423f-a3c2-e1dcdd55e932 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:10.662604 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1603e96-2f36-43ef-b8de-d2eec439dc64 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:10.676911 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:10.709542 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cad92-e084-4a63-9ec0-7ec0884fbc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:10.710843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cad92-e084-4a63-9ec0-7ec0884fbc77 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:10.711286 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cad92-e084-4a63-9ec0-7ec0884fbc77 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:10.711720 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cad92-e084-4a63-9ec0-7ec0884fbc77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:10.729132 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7cad92-e084-4a63-9ec0-7ec0884fbc77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:10.730923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7cad92-e084-4a63-9ec0-7ec0884fbc77 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:10.750194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-cc7cad92-e084-4a63-9ec0-7ec0884fbc77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:10.750194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:10.750194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:10.750194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:10.750194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:10.750194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:10.750194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:10.750194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:10.750194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:10.750194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:10.750194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:10.750194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:10.750194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:10.750194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:10.750194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:10.751341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:10.751341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:10.751341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:10.751341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:10.751341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:10.751341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:10.751341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:10.751341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:10.751341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:10.751341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:10.751341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:10.751341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:10.751341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:10.751341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:10.751341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:10.751341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:10.751341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:10.752400 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:10.752400 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:10.752400 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:10.752400 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:10.760553 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 239/492] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:10.762311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cad92-e084-4a63-9ec0-7ec0884fbc77 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:10.771757 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39cb31-aac3-423f-a3c2-e1dcdd55e932 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:10.804846 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:10.813071 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca775aa2-692b-48b2-886e-ce29868eb7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:10.813755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca775aa2-692b-48b2-886e-ce29868eb7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:10.814042 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca775aa2-692b-48b2-886e-ce29868eb7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:10.814325 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca775aa2-692b-48b2-886e-ce29868eb7c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:10.828771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca775aa2-692b-48b2-886e-ce29868eb7c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:10.830304 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca775aa2-692b-48b2-886e-ce29868eb7c7 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:10.847917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-ca775aa2-692b-48b2-886e-ce29868eb7c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:10.847917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:10.847917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:10.847917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:10.847917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:10.847917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:10.847917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:10.847917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:10.847917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:10.847917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:10.847917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:10.847917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:10.847917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:10.847917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:10.847917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:10.850019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:10.850019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:10.850019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:10.850019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:10.850019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:10.850019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:10.850019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:10.850019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:10.850019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:10.850019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:10.850019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:10.850019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:10.850019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:10.850019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:10.850019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:10.850019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:10.850019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:10.851995 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:10.851995 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:10.851995 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:10.851995 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:10.851995 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 254/493] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:10.851995 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca775aa2-692b-48b2-886e-ce29868eb7c7 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:10.958337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:10.965116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cad92-e084-4a63-9ec0-7ec0884fbc77 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:10.965315 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca775aa2-692b-48b2-886e-ce29868eb7c7 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:11.005702 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abb5d0-6c9f-42f2-9088-fef798c42164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:11.006443 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abb5d0-6c9f-42f2-9088-fef798c42164 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:11.006669 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abb5d0-6c9f-42f2-9088-fef798c42164 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:11.006953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abb5d0-6c9f-42f2-9088-fef798c42164 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:11.007665 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abb5d0-6c9f-42f2-9088-fef798c42164 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:11.008004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abb5d0-6c9f-42f2-9088-fef798c42164 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:11.009439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abb5d0-6c9f-42f2-9088-fef798c42164 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:11.024705 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 255/494] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:11.025834 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea173240-aa03-4ce8-9dca-a2852a7b41a2 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:11.026121 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea173240-aa03-4ce8-9dca-a2852a7b41a2 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:11.026364 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea173240-aa03-4ce8-9dca-a2852a7b41a2 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:11.032588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 256/495] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:11.032588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea173240-aa03-4ce8-9dca-a2852a7b41a2 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:11.034283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e9d5b-176e-40ce-9b86-71fb82763d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:11.035449 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e9d5b-176e-40ce-9b86-71fb82763d7a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:11.035924 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e9d5b-176e-40ce-9b86-71fb82763d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:11.039518 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e9d5b-176e-40ce-9b86-71fb82763d7a None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:11.040981 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e9d5b-176e-40ce-9b86-71fb82763d7a None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:11.041533 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e9d5b-176e-40ce-9b86-71fb82763d7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:11.044261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e9d5b-176e-40ce-9b86-71fb82763d7a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:11.061189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 240/496] 10.222.0.121 () {56 vars in 1108 bytes} [Thu Aug 4 10:19:10 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:11.062416 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e9d5b-176e-40ce-9b86-71fb82763d7a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:11.150359 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea173240-aa03-4ce8-9dca-a2852a7b41a2 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:11.156051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:11.177142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e9d5b-176e-40ce-9b86-71fb82763d7a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:11.192076 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96504160-c4ba-46e1-878b-58e5dc5dda15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:11.192762 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96504160-c4ba-46e1-878b-58e5dc5dda15 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:11.193054 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96504160-c4ba-46e1-878b-58e5dc5dda15 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:11.193338 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96504160-c4ba-46e1-878b-58e5dc5dda15 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:11.194167 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96504160-c4ba-46e1-878b-58e5dc5dda15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:11.194479 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96504160-c4ba-46e1-878b-58e5dc5dda15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:11.195972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96504160-c4ba-46e1-878b-58e5dc5dda15 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:11.208959 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 257/497] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:11 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:11.210077 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc6ce6-1ff7-428d-b3f0-614ec3b885ff None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:11.210448 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc6ce6-1ff7-428d-b3f0-614ec3b885ff None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:11.210728 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc6ce6-1ff7-428d-b3f0-614ec3b885ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:11.218597 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68522332-673b-487c-b11e-f3d6928f2677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:11.219272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68522332-673b-487c-b11e-f3d6928f2677 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:11.219556 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68522332-673b-487c-b11e-f3d6928f2677 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:11.219836 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68522332-673b-487c-b11e-f3d6928f2677 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:11.228370 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68522332-673b-487c-b11e-f3d6928f2677 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:11.229780 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-fbfc6ce6-1ff7-428d-b3f0-614ec3b885ff None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:11.229951 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68522332-673b-487c-b11e-f3d6928f2677 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:11.231012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-fbfc6ce6-1ff7-428d-b3f0-614ec3b885ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:11.242803 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:11.243852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-68522332-673b-487c-b11e-f3d6928f2677 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 10:19:11.243852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:11.243852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:11.243852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:11.243852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:11.243852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:11.243852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:11.243852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:11.243852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:11.243852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:11.243852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:11.243852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:11.243852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:19:11.243852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:19:11.243852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:19:11.243852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:19:11.245207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:11.245207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:11.245207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:19:11.245207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:11.245207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:19:11.245207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:11.245207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:11.245207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:11.245207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:11.245207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:11.245207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:11.245207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:11.245207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:11.245207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:11.245207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:11.245207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:11.245207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:11.245207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:11.248956 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:11.248956 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:11.248956 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:19:11.248956 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:19:11.248956 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:19:11.248956 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:19:11.248956 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:19:11.248956 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:19:11.248956 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 10:19:11.248956 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:11.248956 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 241/498] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:19:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:11.248956 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68522332-673b-487c-b11e-f3d6928f2677 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:11.308977 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 258/499] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:11 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:19:11.310424 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25dcaf-1f60-4c72-bf24-d4fdb849abb2 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:11.310712 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25dcaf-1f60-4c72-bf24-d4fdb849abb2 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:11.310990 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25dcaf-1f60-4c72-bf24-d4fdb849abb2 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:11.312618 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 259/500] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:11.317127 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25dcaf-1f60-4c72-bf24-d4fdb849abb2 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:11.356938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68522332-673b-487c-b11e-f3d6928f2677 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:11.394654 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0914d-7d4c-4a6c-94ea-0db5d66b79e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:11.395327 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0914d-7d4c-4a6c-94ea-0db5d66b79e5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:11.395614 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0914d-7d4c-4a6c-94ea-0db5d66b79e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:11.395905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0914d-7d4c-4a6c-94ea-0db5d66b79e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:11.407692 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0914d-7d4c-4a6c-94ea-0db5d66b79e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:11.409247 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0914d-7d4c-4a6c-94ea-0db5d66b79e5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:11.411260 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:11.425623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-0de0914d-7d4c-4a6c-94ea-0db5d66b79e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:11.425623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:11.425623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:11.425623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:11.425623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:11.425623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:11.425623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:11.425623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:11.425623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:11.425623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:11.425623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:11.425623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:11.425623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:11.425623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:11.425623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:11.427450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:11.427450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:11.427450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:11.427450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:11.427450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:11.427450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:11.427450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:11.427450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:11.427450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:11.427450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:11.427450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:11.427450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:11.427450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:11.427450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:11.427450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:11.427450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:11.427450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:11.428874 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:11.428874 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:11.428874 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:11.428874 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:11.428874 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 242/501] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:11.428874 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0914d-7d4c-4a6c-94ea-0db5d66b79e5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:11.434716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25dcaf-1f60-4c72-bf24-d4fdb849abb2 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:11.471574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57479c39-56a5-4d15-ab38-20bd5c758091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:11.473038 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57479c39-56a5-4d15-ab38-20bd5c758091 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:11.473038 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57479c39-56a5-4d15-ab38-20bd5c758091 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:11.473038 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57479c39-56a5-4d15-ab38-20bd5c758091 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:11.481061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57479c39-56a5-4d15-ab38-20bd5c758091 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:11.482596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57479c39-56a5-4d15-ab38-20bd5c758091 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:11.497136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-57479c39-56a5-4d15-ab38-20bd5c758091 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:19:11.497136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:11.497136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:11.497136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:11.497136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:11.497136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:11.497136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:11.497136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:11.497136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:11.497136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:11.497136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:11.497136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:11.497136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:19:11.497136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:19:11.497136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:19:11.497136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:19:11.498249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:11.498249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:11.498249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:19:11.498249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:11.498249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:19:11.498249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:11.498249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:11.498249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:11.498249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:11.498249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:11.498249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:11.498249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:11.498249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:11.498249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:11.498249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:11.498249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:11.498249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:11.498249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:11.499377 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:11.499377 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:11.499377 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:19:11.499377 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:19:11.499377 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:19:11.499377 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:19:11.499377 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:19:11.499377 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:19:11.499377 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:19:11.499377 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:11.499377 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 260/502] 10.222.0.121 () {56 vars in 1098 bytes} [Thu Aug 4 10:19:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:11.500908 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0f3a1-83e5-46a6-aba6-3934b862cceb None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:11.500908 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0f3a1-83e5-46a6-aba6-3934b862cceb None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:11.500908 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0f3a1-83e5-46a6-aba6-3934b862cceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:11.515247 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:11.521487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-b5b0f3a1-83e5-46a6-aba6-3934b862cceb None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:11.522818 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-b5b0f3a1-83e5-46a6-aba6-3934b862cceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:11.575275 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0914d-7d4c-4a6c-94ea-0db5d66b79e5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:11.601954 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 261/503] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:11 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:19:11.602868 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0f3a1-83e5-46a6-aba6-3934b862cceb None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:11.646043 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102d027-ba8e-4c53-acad-b32346e8fce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:11.646691 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102d027-ba8e-4c53-acad-b32346e8fce6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:11.646983 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102d027-ba8e-4c53-acad-b32346e8fce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:11.647253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102d027-ba8e-4c53-acad-b32346e8fce6 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:11.648103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102d027-ba8e-4c53-acad-b32346e8fce6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:11.648411 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102d027-ba8e-4c53-acad-b32346e8fce6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:11.649894 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102d027-ba8e-4c53-acad-b32346e8fce6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:11.709941 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 243/504] 10.222.0.121 () {56 vars in 1142 bytes} [Thu Aug 4 10:19:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:11.710810 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102d027-ba8e-4c53-acad-b32346e8fce6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:11.718735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0f3a1-83e5-46a6-aba6-3934b862cceb None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:11.755268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98d62a-e749-46d7-bf8d-76423e9dd838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:11.755840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98d62a-e749-46d7-bf8d-76423e9dd838 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:11.756164 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98d62a-e749-46d7-bf8d-76423e9dd838 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:11.756470 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98d62a-e749-46d7-bf8d-76423e9dd838 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:11.759363 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:11.767696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:11.769019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98d62a-e749-46d7-bf8d-76423e9dd838 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:11.774259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-ac98d62a-e749-46d7-bf8d-76423e9dd838 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:11.774259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:11.774259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:11.774259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:11.774259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:11.774259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:11.774259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:11.774259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:11.774259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:11.774259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:11.774259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:11.774259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:11.774259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:11.774259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:11.774259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:11.774259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:11.775355 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:11.775355 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:11.775355 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:11.775355 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:11.775355 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:11.775355 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:11.775355 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:11.775355 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:11.775355 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:11.775355 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:11.775355 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:11.775355 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:11.775355 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:11.775355 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:11.775355 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:11.775355 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:11.776243 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:11.776243 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:11.776243 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:11.776243 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:11.776243 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:11.776243 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:11.776243 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:11.776243 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:11.776243 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:11.776243 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:11.776243 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:11.776243 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:11.776243 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 262/505] 10.222.0.121 () {56 vars in 1107 bytes} [Thu Aug 4 10:19:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:11.776919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98d62a-e749-46d7-bf8d-76423e9dd838 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:11.823257 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102d027-ba8e-4c53-acad-b32346e8fce6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:11.860022 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d41eec-0747-4581-97a8-46867d011e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:11.860682 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d41eec-0747-4581-97a8-46867d011e97 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:11.860994 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d41eec-0747-4581-97a8-46867d011e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:11.861238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d41eec-0747-4581-97a8-46867d011e97 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:11.861981 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d41eec-0747-4581-97a8-46867d011e97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:11.862296 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d41eec-0747-4581-97a8-46867d011e97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:11.864135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d41eec-0747-4581-97a8-46867d011e97 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:11.877027 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 244/506] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:11 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:11.878329 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d41eec-0747-4581-97a8-46867d011e97 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:11.884987 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98d62a-e749-46d7-bf8d-76423e9dd838 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:11.928484 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2bf62-2025-474d-91c9-bbcb9b2d81b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:11.929117 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2bf62-2025-474d-91c9-bbcb9b2d81b8 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:11.929397 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2bf62-2025-474d-91c9-bbcb9b2d81b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:11.929665 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2bf62-2025-474d-91c9-bbcb9b2d81b8 None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:11.930477 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2bf62-2025-474d-91c9-bbcb9b2d81b8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:11.930782 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2bf62-2025-474d-91c9-bbcb9b2d81b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:11.932274 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2bf62-2025-474d-91c9-bbcb9b2d81b8 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:11.974845 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 263/507] 10.222.0.121 () {56 vars in 1138 bytes} [Thu Aug 4 10:19:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:11.974845 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2bf62-2025-474d-91c9-bbcb9b2d81b8 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:11.994366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d41eec-0747-4581-97a8-46867d011e97 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:12.011213 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:12.033533 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51267ed-1bd7-4394-902b-7e7e4fc78b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:12.034194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51267ed-1bd7-4394-902b-7e7e4fc78b09 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:12.034481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51267ed-1bd7-4394-902b-7e7e4fc78b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:12.034763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51267ed-1bd7-4394-902b-7e7e4fc78b09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:12.035333 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:12.046690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51267ed-1bd7-4394-902b-7e7e4fc78b09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:12.048401 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51267ed-1bd7-4394-902b-7e7e4fc78b09 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:12.064782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-a51267ed-1bd7-4394-902b-7e7e4fc78b09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:12.064782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:12.064782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:12.064782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:12.064782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:12.064782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:12.064782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:12.064782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:12.064782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:12.064782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:12.064782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:12.064782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:12.064782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:12.064782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:12.064782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:12.065809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:12.065809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:12.065809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:12.065809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:12.065809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:12.065809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:12.065809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:12.065809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:12.065809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:12.065809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:12.065809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:12.065809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:12.065809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:12.065809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:12.065809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:12.065809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:12.065809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:12.066730 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:12.066730 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:12.066730 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:12.066730 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:12.066730 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 245/508] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:12.067025 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51267ed-1bd7-4394-902b-7e7e4fc78b09 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:12.087819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2bf62-2025-474d-91c9-bbcb9b2d81b8 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:12.123698 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be878a59-515e-4c29-a5e0-9b5c7c93efb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:12.124372 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be878a59-515e-4c29-a5e0-9b5c7c93efb7 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:12.124650 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be878a59-515e-4c29-a5e0-9b5c7c93efb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:12.124921 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be878a59-515e-4c29-a5e0-9b5c7c93efb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:12.179355 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51267ed-1bd7-4394-902b-7e7e4fc78b09 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:12.199297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be878a59-515e-4c29-a5e0-9b5c7c93efb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:12.200869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be878a59-515e-4c29-a5e0-9b5c7c93efb7 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:12.216351 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce088e29-49fa-46a5-9437-98e4e5c94ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:12.217273 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce088e29-49fa-46a5-9437-98e4e5c94ecc None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:12.217729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce088e29-49fa-46a5-9437-98e4e5c94ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:12.218179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce088e29-49fa-46a5-9437-98e4e5c94ecc None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:12.219371 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce088e29-49fa-46a5-9437-98e4e5c94ecc None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:12.219883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce088e29-49fa-46a5-9437-98e4e5c94ecc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:12.222295 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce088e29-49fa-46a5-9437-98e4e5c94ecc None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:12.241403 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 246/509] 10.222.0.121 () {56 vars in 1108 bytes} [Thu Aug 4 10:19:12 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:12.241403 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce088e29-49fa-46a5-9437-98e4e5c94ecc None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:12.267096 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:12.275168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:12.331611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 264/510] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:19:11 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:19:12.332557 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be878a59-515e-4c29-a5e0-9b5c7c93efb7 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:12.351900 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce088e29-49fa-46a5-9437-98e4e5c94ecc None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:12.391047 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7b3c9-33e3-4ef3-9637-795f4825cdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:12.391687 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7b3c9-33e3-4ef3-9637-795f4825cdda None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:12.392021 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7b3c9-33e3-4ef3-9637-795f4825cdda None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:12.392284 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7b3c9-33e3-4ef3-9637-795f4825cdda None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:12.402492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7b3c9-33e3-4ef3-9637-795f4825cdda None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:12.404021 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7b3c9-33e3-4ef3-9637-795f4825cdda None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:12.416110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-aac7b3c9-33e3-4ef3-9637-795f4825cdda None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 10:19:12.416110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:12.416110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:12.416110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:12.416110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:12.416110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:12.416110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:12.416110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:12.416110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:12.416110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:12.416110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:12.416110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:12.416110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:19:12.416110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:19:12.416110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:19:12.416110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:19:12.417159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:12.417159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:12.417159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:19:12.417159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:12.417159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:19:12.417159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:12.417159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:12.417159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:12.417159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:12.417159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:12.417159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:12.417159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:12.417159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:12.417159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:12.417159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:12.417159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:12.417159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:12.417159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:12.418092 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:12.418092 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:12.418092 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:19:12.418092 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:19:12.418092 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:19:12.418092 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:19:12.418092 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:19:12.418092 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:19:12.418092 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 10:19:12.418092 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:12.418619 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 247/511] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:19:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:12.419528 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7b3c9-33e3-4ef3-9637-795f4825cdda None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:12.445370 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be878a59-515e-4c29-a5e0-9b5c7c93efb7 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:12.479979 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d56ff7c-88f4-4f79-831d-78421fac48a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:12.480633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d56ff7c-88f4-4f79-831d-78421fac48a8 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:12.480911 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d56ff7c-88f4-4f79-831d-78421fac48a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:12.481184 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d56ff7c-88f4-4f79-831d-78421fac48a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:12.506267 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:12.511510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:12.528233 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7b3c9-33e3-4ef3-9637-795f4825cdda None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:12.559525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d56ff7c-88f4-4f79-831d-78421fac48a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:12.561128 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d56ff7c-88f4-4f79-831d-78421fac48a8 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:12.584375 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6384a1b-a634-4930-a7e4-19be0f8a6c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:12.585068 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6384a1b-a634-4930-a7e4-19be0f8a6c51 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:12.585357 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6384a1b-a634-4930-a7e4-19be0f8a6c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:12.585635 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6384a1b-a634-4930-a7e4-19be0f8a6c51 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:12.586374 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6384a1b-a634-4930-a7e4-19be0f8a6c51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:12.586687 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6384a1b-a634-4930-a7e4-19be0f8a6c51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:12.588182 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6384a1b-a634-4930-a7e4-19be0f8a6c51 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:12.602181 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 248/512] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:12 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:12.604395 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6384a1b-a634-4930-a7e4-19be0f8a6c51 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:12.702536 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 265/513] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:19:12 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:19:12.703435 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d56ff7c-88f4-4f79-831d-78421fac48a8 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:12.742237 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6384a1b-a634-4930-a7e4-19be0f8a6c51 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:12.763645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:12.791169 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098473a2-e185-45ef-bb9c-cad2ea068fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:12.791726 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098473a2-e185-45ef-bb9c-cad2ea068fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:12.792041 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098473a2-e185-45ef-bb9c-cad2ea068fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:12.792316 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098473a2-e185-45ef-bb9c-cad2ea068fd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:12.803526 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098473a2-e185-45ef-bb9c-cad2ea068fd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:12.805926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098473a2-e185-45ef-bb9c-cad2ea068fd0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:12.814443 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:12.823918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d56ff7c-88f4-4f79-831d-78421fac48a8 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:12.825217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-098473a2-e185-45ef-bb9c-cad2ea068fd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:12.825217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:12.825217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:12.825217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:12.825217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:12.825217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:12.825217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:12.825217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:12.825217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:12.825217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:12.825217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:12.825217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:12.825217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:12.825217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:12.825217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:12.826140 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:12.826140 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:12.826140 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:12.826140 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:12.826140 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:12.826140 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:12.826140 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:12.826140 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:12.826140 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:12.826140 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:12.826140 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:12.826140 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:12.826140 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:12.826140 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:12.826140 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:12.826140 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:12.826140 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:12.827023 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:12.827023 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:12.827023 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:12.827023 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:12.827023 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 249/514] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:12.827292 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098473a2-e185-45ef-bb9c-cad2ea068fd0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:12.869137 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a0512-07c4-49ea-8381-b672a8738c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:12.869806 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a0512-07c4-49ea-8381-b672a8738c0b None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:12.870097 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a0512-07c4-49ea-8381-b672a8738c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:12.870368 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a0512-07c4-49ea-8381-b672a8738c0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:12.883576 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a0512-07c4-49ea-8381-b672a8738c0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:12.885678 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a0512-07c4-49ea-8381-b672a8738c0b None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:12.903302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-8b4a0512-07c4-49ea-8381-b672a8738c0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:12.903302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:12.903302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:12.903302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:12.903302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:12.903302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:12.903302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:12.903302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:12.903302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:12.903302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:12.903302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:12.903302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:12.903302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:12.903302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:12.903302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:12.909749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:12.909749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:12.909749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:12.909749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:12.909749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:12.909749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:12.909749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:12.909749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:12.909749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:12.909749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:12.909749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:12.909749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:12.909749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:12.909749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:12.909749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:12.909749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:12.909749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:12.910892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:12.910892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:12.910892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:12.910892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:12.910892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 266/515] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:12.910892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a0512-07c4-49ea-8381-b672a8738c0b None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:12.997134 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098473a2-e185-45ef-bb9c-cad2ea068fd0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:13.046981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a0512-07c4-49ea-8381-b672a8738c0b None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:13.048105 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8070e35-3079-4007-8232-30b392c358ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:13.048993 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8070e35-3079-4007-8232-30b392c358ab None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:13.049736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8070e35-3079-4007-8232-30b392c358ab None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:13.050243 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8070e35-3079-4007-8232-30b392c358ab None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:13.051785 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8070e35-3079-4007-8232-30b392c358ab None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:13.052377 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8070e35-3079-4007-8232-30b392c358ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:13.054903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8070e35-3079-4007-8232-30b392c358ab None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:13.055499 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:13.085405 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 250/516] 10.222.0.121 () {56 vars in 1142 bytes} [Thu Aug 4 10:19:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:13.085405 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8070e35-3079-4007-8232-30b392c358ab None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:13.088684 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c821de-5829-40b3-a2b4-cfdc7bd260a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:13.089338 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c821de-5829-40b3-a2b4-cfdc7bd260a0 None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:13.090178 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c821de-5829-40b3-a2b4-cfdc7bd260a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:13.090438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c821de-5829-40b3-a2b4-cfdc7bd260a0 None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:13.091147 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c821de-5829-40b3-a2b4-cfdc7bd260a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:13.093080 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c821de-5829-40b3-a2b4-cfdc7bd260a0 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:13.120238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 267/517] 10.222.0.121 () {60 vars in 1137 bytes} [Thu Aug 4 10:19:12 2022] POST /identity/v3/users => generated 248 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:19:13.121133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c821de-5829-40b3-a2b4-cfdc7bd260a0 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:13.168498 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:13.233098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8070e35-3079-4007-8232-30b392c358ab None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:13.253704 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c821de-5829-40b3-a2b4-cfdc7bd260a0 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:13.279285 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df0651-a669-4a01-aa47-ed417c5f9479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:13.279851 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df0651-a669-4a01-aa47-ed417c5f9479 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:13.280176 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df0651-a669-4a01-aa47-ed417c5f9479 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:13.280522 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df0651-a669-4a01-aa47-ed417c5f9479 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:13.296080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df0651-a669-4a01-aa47-ed417c5f9479 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:13.297776 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df0651-a669-4a01-aa47-ed417c5f9479 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:13.300659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27fcd0-3ec9-4e1d-9d9a-48fbd418b865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:13.301409 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27fcd0-3ec9-4e1d-9d9a-48fbd418b865 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:13.301704 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27fcd0-3ec9-4e1d-9d9a-48fbd418b865 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:13.301991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27fcd0-3ec9-4e1d-9d9a-48fbd418b865 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:13.302722 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e27fcd0-3ec9-4e1d-9d9a-48fbd418b865 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:13.303166 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e27fcd0-3ec9-4e1d-9d9a-48fbd418b865 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:13.304660 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e27fcd0-3ec9-4e1d-9d9a-48fbd418b865 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:13.316976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-b9df0651-a669-4a01-aa47-ed417c5f9479 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:13.316976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:13.316976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:13.316976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:13.316976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:13.316976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:13.316976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:13.316976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:13.316976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:13.316976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:13.316976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:13.316976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:13.316976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:13.316976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:13.316976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:13.317976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:13.317976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:13.317976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:13.317976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:13.317976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:13.317976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:13.317976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:13.317976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:13.317976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:13.317976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:13.317976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:13.317976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:13.317976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:13.317976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:13.317976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:13.317976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:13.317976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:13.319440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:13.319440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:13.319440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:13.319440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:13.319440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 251/518] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:13.319440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df0651-a669-4a01-aa47-ed417c5f9479 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:13.323929 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 268/519] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:13.324883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27fcd0-3ec9-4e1d-9d9a-48fbd418b865 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:13.362501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:13.461879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df0651-a669-4a01-aa47-ed417c5f9479 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:13.462370 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27fcd0-3ec9-4e1d-9d9a-48fbd418b865 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:13.476693 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:13.510491 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ce781-d4e4-417d-8876-5724de34cf9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:13.511140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ce781-d4e4-417d-8876-5724de34cf9c None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:13.511423 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ce781-d4e4-417d-8876-5724de34cf9c None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:13.511706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ce781-d4e4-417d-8876-5724de34cf9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:13.514038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5ef70-8d1e-4fbd-84eb-8c0c46555362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:13.514772 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5ef70-8d1e-4fbd-84eb-8c0c46555362 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:13.515138 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5ef70-8d1e-4fbd-84eb-8c0c46555362 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:13.515484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5ef70-8d1e-4fbd-84eb-8c0c46555362 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:13.516668 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5ef70-8d1e-4fbd-84eb-8c0c46555362 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:13.516831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5ef70-8d1e-4fbd-84eb-8c0c46555362 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:13.519199 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5ef70-8d1e-4fbd-84eb-8c0c46555362 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:13.540338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 252/520] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:13.541321 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5ef70-8d1e-4fbd-84eb-8c0c46555362 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:13.608435 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ce781-d4e4-417d-8876-5724de34cf9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:13.611324 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ce781-d4e4-417d-8876-5724de34cf9c None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:13.652221 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:13.684429 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5ef70-8d1e-4fbd-84eb-8c0c46555362 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:13.721381 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6b0aa-2314-445a-ae9b-cac32f2ac7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:13.722016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6b0aa-2314-445a-ae9b-cac32f2ac7f5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:13.722310 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6b0aa-2314-445a-ae9b-cac32f2ac7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:13.722591 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6b0aa-2314-445a-ae9b-cac32f2ac7f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:13.723297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6b0aa-2314-445a-ae9b-cac32f2ac7f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:13.723609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6b0aa-2314-445a-ae9b-cac32f2ac7f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:13.725123 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6b0aa-2314-445a-ae9b-cac32f2ac7f5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:13.737129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 253/521] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:13.737778 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6b0aa-2314-445a-ae9b-cac32f2ac7f5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:13.752302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 269/522] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:19:13 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:19:13.752302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ce781-d4e4-417d-8876-5724de34cf9c None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:13.769577 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:13.847256 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6b0aa-2314-445a-ae9b-cac32f2ac7f5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:13.863737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ce781-d4e4-417d-8876-5724de34cf9c None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:13.888637 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:13.889349 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6bd2b0-f8cd-4994-ab2e-d425c5316d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:13.890216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bd2b0-f8cd-4994-ab2e-d425c5316d19 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:13.890667 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bd2b0-f8cd-4994-ab2e-d425c5316d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:13.891115 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bd2b0-f8cd-4994-ab2e-d425c5316d19 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:13.899616 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6bd2b0-f8cd-4994-ab2e-d425c5316d19 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:13.901154 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6bd2b0-f8cd-4994-ab2e-d425c5316d19 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:13.902863 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777093c-f7ac-4c84-b4b0-7fb95fc9ab1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:13.903547 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777093c-f7ac-4c84-b4b0-7fb95fc9ab1d None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:13.903830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777093c-f7ac-4c84-b4b0-7fb95fc9ab1d None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:13.904158 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777093c-f7ac-4c84-b4b0-7fb95fc9ab1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:13.911849 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a777093c-f7ac-4c84-b4b0-7fb95fc9ab1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:13.913463 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a777093c-f7ac-4c84-b4b0-7fb95fc9ab1d None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:13.913926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-2c6bd2b0-f8cd-4994-ab2e-d425c5316d19 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 10:19:13.913926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:13.913926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:13.913926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:13.913926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:13.913926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:13.913926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:13.913926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:13.913926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:13.913926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:13.913926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:13.913926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:13.913926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:19:13.913926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:19:13.913926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:19:13.913926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:19:13.915159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:13.915159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:13.915159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:19:13.915159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:13.915159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:19:13.915159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:13.915159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:13.915159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:13.915159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:13.915159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:13.915159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:13.915159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:13.915159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:13.915159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:13.915159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:13.915159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:13.915159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:13.915159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:13.916478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:13.916478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:13.916478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:19:13.916478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:19:13.916478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:19:13.916478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:19:13.916478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:19:13.916478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:19:13.916478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 10:19:13.916478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:13.919999 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 254/523] 10.222.0.121 () {56 vars in 1107 bytes} [Thu Aug 4 10:19:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:13.920897 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6bd2b0-f8cd-4994-ab2e-d425c5316d19 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:13.931738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-a777093c-f7ac-4c84-b4b0-7fb95fc9ab1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:13.931738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:13.931738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:13.931738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:13.931738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:13.931738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:13.931738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:13.931738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:13.931738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:13.931738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:13.931738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:13.931738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:13.931738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:19:13.931738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:19:13.931738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:19:13.932948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:19:13.932948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:13.932948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:13.932948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:13.932948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:13.932948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:13.932948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:13.932948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:13.932948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:13.932948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:13.932948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:13.932948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:13.932948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:13.932948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:13.932948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:13.932948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:13.932948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:13.933845 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:19:13.933845 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:19:13.933845 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:19:13.933845 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:19:13.933845 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:19:13.933845 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:19:13.933845 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:13.933845 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:13.933845 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 270/524] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:13.935830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777093c-f7ac-4c84-b4b0-7fb95fc9ab1d None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:14.032899 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:14.051823 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6bd2b0-f8cd-4994-ab2e-d425c5316d19 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:14.068200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777093c-f7ac-4c84-b4b0-7fb95fc9ab1d None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:14.094328 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bca060-a9c0-4af9-a557-672752f57d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:14.094989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bca060-a9c0-4af9-a557-672752f57d40 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:14.095275 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bca060-a9c0-4af9-a557-672752f57d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:14.095630 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bca060-a9c0-4af9-a557-672752f57d40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:14.103009 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bca060-a9c0-4af9-a557-672752f57d40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:14.106176 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bca060-a9c0-4af9-a557-672752f57d40 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:14.106326 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c19cc2-aa69-4983-9c03-5f49d5557481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:14.106326 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c19cc2-aa69-4983-9c03-5f49d5557481 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:14.106326 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c19cc2-aa69-4983-9c03-5f49d5557481 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:14.106326 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c19cc2-aa69-4983-9c03-5f49d5557481 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:14.121308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c19cc2-aa69-4983-9c03-5f49d5557481 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:14.122812 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c19cc2-aa69-4983-9c03-5f49d5557481 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:14.123766 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-d7bca060-a9c0-4af9-a557-672752f57d40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:14.123766 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:14.123766 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:14.123766 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:14.123766 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:14.123766 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:14.123766 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:14.123766 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:14.123766 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:14.123766 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:14.123766 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:14.123766 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:14.123766 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:19:14.123766 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:19:14.123766 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:19:14.124796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:19:14.124796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:14.124796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:14.124796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:14.124796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:14.124796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:14.124796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:14.124796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:14.124796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:14.124796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:14.124796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:14.124796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:14.124796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:14.124796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:14.124796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:14.124796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:14.124796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:14.125732 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:19:14.125732 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:19:14.125732 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:19:14.125732 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:19:14.125732 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:19:14.125732 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:19:14.125732 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:14.125732 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:14.125732 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 255/525] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:14.127138 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bca060-a9c0-4af9-a557-672752f57d40 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:14.144960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-c6c19cc2-aa69-4983-9c03-5f49d5557481 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:14.144960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:14.144960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:14.144960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:14.144960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:14.144960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:14.144960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:14.144960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:14.144960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:14.144960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:14.144960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:14.144960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:14.144960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:14.144960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:14.144960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:14.146109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:14.146109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:14.146109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:14.146109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:14.146109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:14.146109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:14.146109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:14.146109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:14.146109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:14.146109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:14.146109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:14.146109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:14.146109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:14.146109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:14.146109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:14.146109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:14.146109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:14.147089 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:14.147089 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:14.147089 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:14.147089 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:14.147089 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 271/526] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:14.147089 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c19cc2-aa69-4983-9c03-5f49d5557481 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:14.188446 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:14.254399 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bca060-a9c0-4af9-a557-672752f57d40 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:14.288900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c19cc2-aa69-4983-9c03-5f49d5557481 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:14.299781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6c205-71cd-41b7-95c4-59eb9e2af8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:14.300485 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6c205-71cd-41b7-95c4-59eb9e2af8fc None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:14.300771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6c205-71cd-41b7-95c4-59eb9e2af8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:14.301058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6c205-71cd-41b7-95c4-59eb9e2af8fc None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:14.301906 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db6c205-71cd-41b7-95c4-59eb9e2af8fc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:14.302228 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db6c205-71cd-41b7-95c4-59eb9e2af8fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:14.303766 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db6c205-71cd-41b7-95c4-59eb9e2af8fc None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:14.323759 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 256/527] 10.222.0.121 () {56 vars in 1144 bytes} [Thu Aug 4 10:19:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:14.325059 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6c205-71cd-41b7-95c4-59eb9e2af8fc None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:14.334991 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:14.338422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bec863-921b-4c26-a4d1-889641543bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:14.339265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bec863-921b-4c26-a4d1-889641543bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:14.339631 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bec863-921b-4c26-a4d1-889641543bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:14.340178 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bec863-921b-4c26-a4d1-889641543bd3 None admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:14.340879 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bec863-921b-4c26-a4d1-889641543bd3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:14.341270 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bec863-921b-4c26-a4d1-889641543bd3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:14.342682 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bec863-921b-4c26-a4d1-889641543bd3 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:14.365425 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 272/528] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:19:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:14.366325 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bec863-921b-4c26-a4d1-889641543bd3 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:14.482283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6c205-71cd-41b7-95c4-59eb9e2af8fc None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:14.503215 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:14.512198 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bec863-921b-4c26-a4d1-889641543bd3 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:14.531285 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a3f04-c620-4e00-9ca7-2c452688a9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:14.532324 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a3f04-c620-4e00-9ca7-2c452688a9c8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:14.532611 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a3f04-c620-4e00-9ca7-2c452688a9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:14.532915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a3f04-c620-4e00-9ca7-2c452688a9c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:14.533667 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a3f04-c620-4e00-9ca7-2c452688a9c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:14.533978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a3f04-c620-4e00-9ca7-2c452688a9c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:14.535471 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a3f04-c620-4e00-9ca7-2c452688a9c8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:14.559309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 257/529] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:19:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:14.561251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a3f04-c620-4e00-9ca7-2c452688a9c8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:14.571048 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61d9e8-1f40-40a8-b68e-4132c202a62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:14.571727 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61d9e8-1f40-40a8-b68e-4132c202a62c None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:14.572192 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61d9e8-1f40-40a8-b68e-4132c202a62c None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:14.572520 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61d9e8-1f40-40a8-b68e-4132c202a62c None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:14.573219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61d9e8-1f40-40a8-b68e-4132c202a62c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:14.573529 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61d9e8-1f40-40a8-b68e-4132c202a62c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:14.575620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61d9e8-1f40-40a8-b68e-4132c202a62c None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:14.597431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 273/530] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:14 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:14.598634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61d9e8-1f40-40a8-b68e-4132c202a62c None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:14.658328 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:14.710333 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a3f04-c620-4e00-9ca7-2c452688a9c8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:14.760978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61d9e8-1f40-40a8-b68e-4132c202a62c None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:14.762213 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07ea33-470f-4c0f-948d-46098924fe59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:14.762864 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07ea33-470f-4c0f-948d-46098924fe59 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:14.763168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07ea33-470f-4c0f-948d-46098924fe59 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:14.763459 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07ea33-470f-4c0f-948d-46098924fe59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:14.815588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c9ad4-81e3-4818-9dcd-1ac919830595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:14.815588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c9ad4-81e3-4818-9dcd-1ac919830595 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:14.815588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c9ad4-81e3-4818-9dcd-1ac919830595 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:14.815588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c9ad4-81e3-4818-9dcd-1ac919830595 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:14.816453 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c9ad4-81e3-4818-9dcd-1ac919830595 None admin] RBAC: Adding query filter params (role.id=2edb704d42834db6acd295fa3a8be5d5, scope.project.id=22768aba760b4307aab131d1312b2bf6, user.id=1b31c86bc9334d2b8acc35a035f90442) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:14.816453 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c9ad4-81e3-4818-9dcd-1ac919830595 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:14.817072 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c9ad4-81e3-4818-9dcd-1ac919830595 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:14.824776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 274/531] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:19:14 2022] GET /identity/v3/role_assignments?user.id=1b31c86bc9334d2b8acc35a035f90442&scope.project.id=22768aba760b4307aab131d1312b2bf6&role.id=2edb704d42834db6acd295fa3a8be5d5 => generated 260 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:14.826177 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c9ad4-81e3-4818-9dcd-1ac919830595 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:14.854233 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:14.879012 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a07ea33-470f-4c0f-948d-46098924fe59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:14.881639 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a07ea33-470f-4c0f-948d-46098924fe59 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:14.985234 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:14.992536 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c9ad4-81e3-4818-9dcd-1ac919830595 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:15.033195 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab633c3-01df-426e-86fd-0d09a1c479cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:15.034340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab633c3-01df-426e-86fd-0d09a1c479cd None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:15.034835 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab633c3-01df-426e-86fd-0d09a1c479cd None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:15.035321 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab633c3-01df-426e-86fd-0d09a1c479cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:15.036441 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab633c3-01df-426e-86fd-0d09a1c479cd None admin] RBAC: Adding query filter params (role.id=2edb704d42834db6acd295fa3a8be5d5, scope.project.id=22768aba760b4307aab131d1312b2bf6, user.id=0d947aeb749546309a8d4b7938087ba0) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:15.036970 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab633c3-01df-426e-86fd-0d09a1c479cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:15.038761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab633c3-01df-426e-86fd-0d09a1c479cd None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:15.045696 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 275/532] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:19:14 2022] GET /identity/v3/role_assignments?user.id=0d947aeb749546309a8d4b7938087ba0&scope.project.id=22768aba760b4307aab131d1312b2bf6&role.id=2edb704d42834db6acd295fa3a8be5d5 => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:15.047122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 258/533] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:19:14 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:19:15.047380 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab633c3-01df-426e-86fd-0d09a1c479cd None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:15.066795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07ea33-470f-4c0f-948d-46098924fe59 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:15.199287 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:15.216417 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07ea33-470f-4c0f-948d-46098924fe59 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:15.242500 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab633c3-01df-426e-86fd-0d09a1c479cd None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:15.261604 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f121522-f693-4de0-a423-99c8c942cf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:15.263393 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f121522-f693-4de0-a423-99c8c942cf91 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:15.264172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f121522-f693-4de0-a423-99c8c942cf91 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:15.264895 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f121522-f693-4de0-a423-99c8c942cf91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:15.279238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f121522-f693-4de0-a423-99c8c942cf91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:15.281660 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f121522-f693-4de0-a423-99c8c942cf91 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:15.284529 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc7490d-8805-4e5a-a07e-278a21b5f220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:15.285222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7490d-8805-4e5a-a07e-278a21b5f220 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:15.285503 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7490d-8805-4e5a-a07e-278a21b5f220 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:15.285779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7490d-8805-4e5a-a07e-278a21b5f220 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:15.297498 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc7490d-8805-4e5a-a07e-278a21b5f220 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:15.299005 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc7490d-8805-4e5a-a07e-278a21b5f220 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:15.302980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-0f121522-f693-4de0-a423-99c8c942cf91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:15.302980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:15.302980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:15.302980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:15.302980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:15.302980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:15.302980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:15.302980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:15.302980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:15.302980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:15.302980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:15.302980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:15.302980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:15.302980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:15.302980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:15.303919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:15.303919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:15.303919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:15.303919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:15.303919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:15.303919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:15.303919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:15.303919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:15.303919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:15.303919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:15.303919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:15.303919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:15.303919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:15.303919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:15.303919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:15.303919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:15.303919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:15.304858 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:15.304858 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:15.304858 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:15.304858 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:15.307251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 259/534] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:15.312705 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f121522-f693-4de0-a423-99c8c942cf91 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:15.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-ebc7490d-8805-4e5a-a07e-278a21b5f220 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:15.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:15.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:15.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:15.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:15.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:15.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:15.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:15.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:15.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:15.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:15.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:15.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:19:15.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:19:15.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:19:15.314693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:19:15.314693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:15.314693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:15.314693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:15.314693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:15.314693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:15.314693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:15.314693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:15.314693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:15.314693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:15.314693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:15.314693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:15.314693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:15.314693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:15.314693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:15.314693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:15.314693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:15.315779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:19:15.315779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:19:15.315779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:19:15.315779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:19:15.315779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:19:15.315779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:19:15.315779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:15.315779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:15.320029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 276/535] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:15.324313 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc7490d-8805-4e5a-a07e-278a21b5f220 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:15.345630 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:15.480829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f121522-f693-4de0-a423-99c8c942cf91 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:15.498137 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc7490d-8805-4e5a-a07e-278a21b5f220 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:15.526404 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c4a2c2-2160-40ec-8130-2b66d5537de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:15.527057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4a2c2-2160-40ec-8130-2b66d5537de5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:15.527343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4a2c2-2160-40ec-8130-2b66d5537de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:15.527719 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4a2c2-2160-40ec-8130-2b66d5537de5 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:15.528619 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c4a2c2-2160-40ec-8130-2b66d5537de5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:15.528926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c4a2c2-2160-40ec-8130-2b66d5537de5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:15.530374 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c4a2c2-2160-40ec-8130-2b66d5537de5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:15.536563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42887e66-cca6-4d20-842d-b690eddd1ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:15.537256 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42887e66-cca6-4d20-842d-b690eddd1ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:15.537549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42887e66-cca6-4d20-842d-b690eddd1ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:15.537835 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42887e66-cca6-4d20-842d-b690eddd1ff1 None admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:15.538611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42887e66-cca6-4d20-842d-b690eddd1ff1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:15.538924 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42887e66-cca6-4d20-842d-b690eddd1ff1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:15.540595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42887e66-cca6-4d20-842d-b690eddd1ff1 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:15.546963 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 260/536] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:15.547380 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36f42f-5bbb-47d2-8d15-5837364fafc9 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:15.547674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36f42f-5bbb-47d2-8d15-5837364fafc9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:15.547989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36f42f-5bbb-47d2-8d15-5837364fafc9 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:15.554837 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 277/538] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:19:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:19:15.555072 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 261/538] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:15.558611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42887e66-cca6-4d20-842d-b690eddd1ff1 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:15.558786 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7cfe3-0df7-4c69-9f5b-65ed8508f604 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:15.558786 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7cfe3-0df7-4c69-9f5b-65ed8508f604 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:15.558786 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7cfe3-0df7-4c69-9f5b-65ed8508f604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:15.559017 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:15.590267 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-28d7cfe3-0df7-4c69-9f5b-65ed8508f604 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:15.591895 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-28d7cfe3-0df7-4c69-9f5b-65ed8508f604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:15.700154 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 262/539] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:15 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:19:15.702484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7cfe3-0df7-4c69-9f5b-65ed8508f604 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:15.710150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42887e66-cca6-4d20-842d-b690eddd1ff1 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:15.740444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:15.767328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47b29b-9ed9-47b6-90b7-1219b81f19fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:15.767967 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47b29b-9ed9-47b6-90b7-1219b81f19fe None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:15.768258 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47b29b-9ed9-47b6-90b7-1219b81f19fe None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:15.768535 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47b29b-9ed9-47b6-90b7-1219b81f19fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:15.780907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de47b29b-9ed9-47b6-90b7-1219b81f19fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:15.782753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de47b29b-9ed9-47b6-90b7-1219b81f19fe None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:15.798978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-de47b29b-9ed9-47b6-90b7-1219b81f19fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:15.798978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:15.798978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:15.798978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:15.798978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:15.798978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:15.798978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:15.798978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:15.798978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:15.798978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:15.798978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:15.798978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:15.798978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:19:15.798978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:19:15.798978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:19:15.800140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:19:15.800140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:15.800140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:15.800140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:15.800140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:15.800140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:15.800140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:15.800140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:15.800140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:15.800140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:15.800140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:15.800140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:15.800140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:15.800140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:15.800140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:15.800140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:15.800140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:15.801018 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:19:15.801018 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:19:15.801018 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:19:15.801018 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:19:15.801018 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:19:15.801018 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:19:15.801018 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:15.801018 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:15.801604 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 278/540] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:15.807983 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47b29b-9ed9-47b6-90b7-1219b81f19fe None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:15.871790 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7cfe3-0df7-4c69-9f5b-65ed8508f604 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:15.901626 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:15.918468 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406c26d-93cc-4caf-985e-702753aae877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:15.919436 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406c26d-93cc-4caf-985e-702753aae877 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:15.919766 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406c26d-93cc-4caf-985e-702753aae877 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:15.920210 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406c26d-93cc-4caf-985e-702753aae877 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:15.921167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406c26d-93cc-4caf-985e-702753aae877 None admin] RBAC: Adding query filter params (role.id=2edb704d42834db6acd295fa3a8be5d5, scope.project.id=22768aba760b4307aab131d1312b2bf6, user.id=c19bb76d0e2544299916007fbfa62578) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:15.921557 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406c26d-93cc-4caf-985e-702753aae877 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:15.923338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406c26d-93cc-4caf-985e-702753aae877 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:15.936638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 263/541] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:19:15 2022] GET /identity/v3/role_assignments?user.id=c19bb76d0e2544299916007fbfa62578&scope.project.id=22768aba760b4307aab131d1312b2bf6&role.id=2edb704d42834db6acd295fa3a8be5d5 => generated 260 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:15.937788 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406c26d-93cc-4caf-985e-702753aae877 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:15.955634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47b29b-9ed9-47b6-90b7-1219b81f19fe None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:16.016193 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc162239-ccbb-4470-9c5e-42086da52d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:16.017647 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc162239-ccbb-4470-9c5e-42086da52d12 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:16.017954 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc162239-ccbb-4470-9c5e-42086da52d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:16.018235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc162239-ccbb-4470-9c5e-42086da52d12 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:16.046930 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc162239-ccbb-4470-9c5e-42086da52d12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:16.058761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-cc162239-ccbb-4470-9c5e-42086da52d12 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:16.058761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:16.058761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:16.058761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:16.058761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:16.058761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:16.058761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:16.058761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:16.058761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:16.058761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:16.058761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:16.058761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:16.058761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:16.058761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:16.058761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:16.058761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:16.060109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:16.060109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:16.060109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:16.060109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:16.060109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:16.060109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:16.060109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:16.060109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:16.060109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:16.060109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:16.060109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:16.060109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:16.060109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:16.060109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:16.060109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:16.060109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:16.061064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:16.061064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:16.061064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:16.061064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:16.061064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:16.061064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:16.061064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:16.061064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:16.061064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:16.061064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:16.061064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:16.061064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:16.061926 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 279/542] 10.222.0.121 () {56 vars in 1107 bytes} [Thu Aug 4 10:19:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:16.068609 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc162239-ccbb-4470-9c5e-42086da52d12 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:16.123922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:16.195045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406c26d-93cc-4caf-985e-702753aae877 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:16.232729 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc162239-ccbb-4470-9c5e-42086da52d12 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:16.246044 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39327a56-f03b-41db-8f17-30224a561007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:16.247198 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39327a56-f03b-41db-8f17-30224a561007 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:16.247726 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39327a56-f03b-41db-8f17-30224a561007 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:16.249442 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39327a56-f03b-41db-8f17-30224a561007 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:16.250484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39327a56-f03b-41db-8f17-30224a561007 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:16.251010 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39327a56-f03b-41db-8f17-30224a561007 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:16.252792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39327a56-f03b-41db-8f17-30224a561007 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:16.267617 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 264/543] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:19:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:16.268156 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:16.274998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39327a56-f03b-41db-8f17-30224a561007 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:16.292681 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35021bc4-d77d-4eb3-99ae-f13d08e51ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:16.293996 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35021bc4-d77d-4eb3-99ae-f13d08e51ccc None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:16.294626 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35021bc4-d77d-4eb3-99ae-f13d08e51ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:16.295090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35021bc4-d77d-4eb3-99ae-f13d08e51ccc None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:16.296050 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35021bc4-d77d-4eb3-99ae-f13d08e51ccc None admin] RBAC: Adding query filter params (name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:16.296514 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35021bc4-d77d-4eb3-99ae-f13d08e51ccc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:16.299094 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35021bc4-d77d-4eb3-99ae-f13d08e51ccc None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:16.318031 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 280/544] 10.222.0.121 () {56 vars in 1108 bytes} [Thu Aug 4 10:19:16 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:16.321389 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35021bc4-d77d-4eb3-99ae-f13d08e51ccc None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:16.395388 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39327a56-f03b-41db-8f17-30224a561007 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:16.429474 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:16.437054 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45ce06-47c3-4406-9d95-2b028927be75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:16.438161 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45ce06-47c3-4406-9d95-2b028927be75 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:16.438685 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45ce06-47c3-4406-9d95-2b028927be75 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:16.439173 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45ce06-47c3-4406-9d95-2b028927be75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:16.440241 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45ce06-47c3-4406-9d95-2b028927be75 None admin] RBAC: Adding query filter params (role.id=2edb704d42834db6acd295fa3a8be5d5, scope.project.id=22768aba760b4307aab131d1312b2bf6, user.id=90c7eaf4500f4ef496ba72405522006d) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:16.440781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45ce06-47c3-4406-9d95-2b028927be75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:16.442511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45ce06-47c3-4406-9d95-2b028927be75 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:16.450011 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 265/545] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:19:16 2022] GET /identity/v3/role_assignments?user.id=90c7eaf4500f4ef496ba72405522006d&scope.project.id=22768aba760b4307aab131d1312b2bf6&role.id=2edb704d42834db6acd295fa3a8be5d5 => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:19:16.496041 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35021bc4-d77d-4eb3-99ae-f13d08e51ccc None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:16.551791 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b1e96-8386-437f-bb12-7f4b840fe99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:16.552502 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b1e96-8386-437f-bb12-7f4b840fe99a None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:16.552708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b1e96-8386-437f-bb12-7f4b840fe99a None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:16.553038 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b1e96-8386-437f-bb12-7f4b840fe99a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:16.569886 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b1e96-8386-437f-bb12-7f4b840fe99a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:16.571531 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b1e96-8386-437f-bb12-7f4b840fe99a None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:16.593259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-0f0b1e96-8386-437f-bb12-7f4b840fe99a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:16.593259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:16.593259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:16.593259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:16.593259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:16.593259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:16.593259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:16.593259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:16.593259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:16.593259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:16.593259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:16.593259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:16.593259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:16.593259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:16.593259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:16.594330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:16.594330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:16.594330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:16.594330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:16.594330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:16.594330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:16.594330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:16.594330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:16.594330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:16.594330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:16.594330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:16.594330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:16.594330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:16.594330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:16.594330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:16.594330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:16.594330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:16.595215 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:16.595215 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:16.595215 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:16.595215 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:16.600202 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 281/546] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:16.604850 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45ce06-47c3-4406-9d95-2b028927be75 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:16.804016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45ce06-47c3-4406-9d95-2b028927be75 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:16.865504 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9550e95-131b-4d6a-839f-f8cd3f207f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:16.866156 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9550e95-131b-4d6a-839f-f8cd3f207f24 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:16.866441 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9550e95-131b-4d6a-839f-f8cd3f207f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:16.866716 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9550e95-131b-4d6a-839f-f8cd3f207f24 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:16.867430 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9550e95-131b-4d6a-839f-f8cd3f207f24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:16.867741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9550e95-131b-4d6a-839f-f8cd3f207f24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:16.869208 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9550e95-131b-4d6a-839f-f8cd3f207f24 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:16.890711 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 266/547] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:16 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:16.897251 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b1e96-8386-437f-bb12-7f4b840fe99a None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:16.938077 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:17.017011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b1e96-8386-437f-bb12-7f4b840fe99a None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:17.055577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d400607-219c-4bd3-a75d-ae56e7b3b533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:17.056233 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d400607-219c-4bd3-a75d-ae56e7b3b533 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:17.056508 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d400607-219c-4bd3-a75d-ae56e7b3b533 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:17.056790 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d400607-219c-4bd3-a75d-ae56e7b3b533 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:17.067963 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d400607-219c-4bd3-a75d-ae56e7b3b533 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:17.069710 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d400607-219c-4bd3-a75d-ae56e7b3b533 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:17.083906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-3d400607-219c-4bd3-a75d-ae56e7b3b533 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 10:19:17.083906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:17.083906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:17.083906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:17.083906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:17.083906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:17.083906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:17.083906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:17.083906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:17.083906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:17.083906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:17.083906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:17.083906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:19:17.083906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:19:17.083906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:19:17.083906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:19:17.085565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:17.085565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:17.085565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:19:17.085565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:17.085565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:19:17.085565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:17.085565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:17.085565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:17.085565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:17.085565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:17.085565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:17.085565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:17.085565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:17.085565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:17.085565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:17.085565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:17.085565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:17.085565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:17.086701 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:17.086701 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:17.086701 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:19:17.086701 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:19:17.086701 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:19:17.086701 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:19:17.086701 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:19:17.086701 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:19:17.086701 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 10:19:17.086701 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:17.090900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 282/548] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:19:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:17.092377 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9550e95-131b-4d6a-839f-f8cd3f207f24 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:17.108220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d0c19-1b3c-43f0-95c0-ba7b4713b0e3 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:17.108805 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d0c19-1b3c-43f0-95c0-ba7b4713b0e3 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:17.108805 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d0c19-1b3c-43f0-95c0-ba7b4713b0e3 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:17.110519 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 283/549] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:17.118417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae56df5-ccfd-492f-ab49-758fc09335ba None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:17.118872 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae56df5-ccfd-492f-ab49-758fc09335ba None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:17.119116 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae56df5-ccfd-492f-ab49-758fc09335ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:17.151476 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-3ae56df5-ccfd-492f-ab49-758fc09335ba None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:17.152862 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-3ae56df5-ccfd-492f-ab49-758fc09335ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:17.213599 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:17.340737 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9550e95-131b-4d6a-839f-f8cd3f207f24 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:17.362101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 284/550] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:17 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:19:17.386763 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae56df5-ccfd-492f-ab49-758fc09335ba None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:17.418469 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:17.471438 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6cc8c6-922c-4a93-a6ac-b8235a207fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:17.472552 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cc8c6-922c-4a93-a6ac-b8235a207fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:17.473314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cc8c6-922c-4a93-a6ac-b8235a207fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:17.473804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cc8c6-922c-4a93-a6ac-b8235a207fa7 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:17.475360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6cc8c6-922c-4a93-a6ac-b8235a207fa7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:17.475920 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6cc8c6-922c-4a93-a6ac-b8235a207fa7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:17.485833 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6cc8c6-922c-4a93-a6ac-b8235a207fa7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:17.506666 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae56df5-ccfd-492f-ab49-758fc09335ba None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:17.547800 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 267/551] 10.222.0.121 () {56 vars in 1140 bytes} [Thu Aug 4 10:19:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 456 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:17.551749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8f19a-3639-42b4-82eb-f4f31cce42c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:17.553450 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8f19a-3639-42b4-82eb-f4f31cce42c8 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:17.553838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8f19a-3639-42b4-82eb-f4f31cce42c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:17.554240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8f19a-3639-42b4-82eb-f4f31cce42c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:17.561962 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6cc8c6-922c-4a93-a6ac-b8235a207fa7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:17.584884 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c8f19a-3639-42b4-82eb-f4f31cce42c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:17.597189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-e0c8f19a-3639-42b4-82eb-f4f31cce42c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:17.597189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:17.597189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:17.597189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:17.597189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:17.597189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:17.597189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:17.597189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:17.597189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:17.597189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:17.597189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:17.597189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:17.597189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:17.597189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:17.597189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:17.597189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:17.599120 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:17.599120 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:17.599120 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:17.599120 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:17.599120 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:17.599120 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:17.599120 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:17.599120 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:17.599120 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:17.599120 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:17.599120 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:17.599120 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:17.599120 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:17.599120 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:17.599120 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:17.599120 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:17.601716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:17.601716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:17.601716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:17.601716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:17.601716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:17.601716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:17.601716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:17.601716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:17.601716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:17.601716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:17.601716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:17.601716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:17.611644 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 285/552] 10.222.0.121 () {56 vars in 1107 bytes} [Thu Aug 4 10:19:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:19:17.617747 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8f19a-3639-42b4-82eb-f4f31cce42c8 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:17.849176 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:17.855264 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6cc8c6-922c-4a93-a6ac-b8235a207fa7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:17.872960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8f19a-3639-42b4-82eb-f4f31cce42c8 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:17.888231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:17.931737 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e388c-709a-42b9-8f97-72d57901a273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:17.933198 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e388c-709a-42b9-8f97-72d57901a273 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:17.933918 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e388c-709a-42b9-8f97-72d57901a273 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:17.934655 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e388c-709a-42b9-8f97-72d57901a273 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:17.941033 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fcc25-12e1-459a-928b-c85bb9342082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:17.943253 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fcc25-12e1-459a-928b-c85bb9342082 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:17.943253 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fcc25-12e1-459a-928b-c85bb9342082 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:17.943253 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fcc25-12e1-459a-928b-c85bb9342082 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:17.944520 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896fcc25-12e1-459a-928b-c85bb9342082 None admin] RBAC: Adding query filter params (name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:17.945051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896fcc25-12e1-459a-928b-c85bb9342082 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:17.947679 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896fcc25-12e1-459a-928b-c85bb9342082 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:17.964709 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 286/553] 10.222.0.121 () {56 vars in 1108 bytes} [Thu Aug 4 10:19:17 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:19:17.967165 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188e190-385e-4804-8a1a-10fb06a6a8b2 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:17.967653 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188e190-385e-4804-8a1a-10fb06a6a8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:17.968425 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188e190-385e-4804-8a1a-10fb06a6a8b2 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:17.979913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 287/554] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:17 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:17.979913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188e190-385e-4804-8a1a-10fb06a6a8b2 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:18.082200 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e388c-709a-42b9-8f97-72d57901a273 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:18.085903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e388c-709a-42b9-8f97-72d57901a273 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:18.171211 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188e190-385e-4804-8a1a-10fb06a6a8b2 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:18.251652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312615b0-05bc-4d64-bfc5-18e15bf62f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:18.253311 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312615b0-05bc-4d64-bfc5-18e15bf62f8e None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:18.254528 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312615b0-05bc-4d64-bfc5-18e15bf62f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:18.255258 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312615b0-05bc-4d64-bfc5-18e15bf62f8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:18.282662 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312615b0-05bc-4d64-bfc5-18e15bf62f8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:18.286261 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312615b0-05bc-4d64-bfc5-18e15bf62f8e None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:18.289214 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 268/555] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:19:17 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 728 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:19:18.291552 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf84dd-6d7f-4d38-a241-f499f9fe3e4b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:18.292177 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf84dd-6d7f-4d38-a241-f499f9fe3e4b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:18.292797 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf84dd-6d7f-4d38-a241-f499f9fe3e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:18.308026 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:18.323839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-312615b0-05bc-4d64-bfc5-18e15bf62f8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:18.323839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:18.323839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:18.323839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:18.323839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:18.323839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:18.323839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:18.323839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:18.323839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:18.323839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:18.323839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:18.323839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:18.323839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:18.323839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:18.323839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:18.325299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:18.325299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:18.325299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:18.325299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:18.325299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:18.325299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:18.325299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:18.325299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:18.325299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:18.325299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:18.325299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:18.325299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:18.325299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:18.325299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:18.325299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:18.325299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:18.325299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:18.326198 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:18.326198 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:18.326198 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:18.326198 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:18.328817 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-52cf84dd-6d7f-4d38-a241-f499f9fe3e4b None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:18.329328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 288/556] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:18.330941 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312615b0-05bc-4d64-bfc5-18e15bf62f8e None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:18.331836 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-52cf84dd-6d7f-4d38-a241-f499f9fe3e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:18.352346 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:18.485134 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 269/557] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:18 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:19:18.487006 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf84dd-6d7f-4d38-a241-f499f9fe3e4b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:18.564573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312615b0-05bc-4d64-bfc5-18e15bf62f8e None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:18.655431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edcd07f-b3f8-42f8-9893-e2d805c3ba37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:18.656692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edcd07f-b3f8-42f8-9893-e2d805c3ba37 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:18.657172 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edcd07f-b3f8-42f8-9893-e2d805c3ba37 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:18.657642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edcd07f-b3f8-42f8-9893-e2d805c3ba37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:18.686650 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edcd07f-b3f8-42f8-9893-e2d805c3ba37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:18.690555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edcd07f-b3f8-42f8-9893-e2d805c3ba37 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:18.713756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf84dd-6d7f-4d38-a241-f499f9fe3e4b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:18.722771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-6edcd07f-b3f8-42f8-9893-e2d805c3ba37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:18.722771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:18.722771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:18.722771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:18.722771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:18.722771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:18.722771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:18.722771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:18.722771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:18.722771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:18.722771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:18.722771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:18.722771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:18.722771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:18.722771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:18.725568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:18.725568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:18.725568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:18.725568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:18.725568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:18.725568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:18.725568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:18.725568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:18.725568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:18.725568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:18.725568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:18.725568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:18.725568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:18.725568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:18.725568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:18.725568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:18.725568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:18.729284 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:18.729284 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:18.729284 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:18.729284 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:18.729284 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 289/558] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:18.729284 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edcd07f-b3f8-42f8-9893-e2d805c3ba37 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:18.790459 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb5001-f822-470f-92b3-884a7c1c7a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:18.790977 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb5001-f822-470f-92b3-884a7c1c7a18 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:18.791316 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb5001-f822-470f-92b3-884a7c1c7a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:18.791578 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb5001-f822-470f-92b3-884a7c1c7a18 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:18.792859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb5001-f822-470f-92b3-884a7c1c7a18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:18.793184 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb5001-f822-470f-92b3-884a7c1c7a18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:18.794817 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb5001-f822-470f-92b3-884a7c1c7a18 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:18.810606 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 270/559] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:18 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:18.811583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb5001-f822-470f-92b3-884a7c1c7a18 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:18.814167 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:18.818298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:18.916735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edcd07f-b3f8-42f8-9893-e2d805c3ba37 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:18.959775 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb5001-f822-470f-92b3-884a7c1c7a18 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:18.983643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbddc7e-0bbc-4d3b-9558-f6bdd7189e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:18.985424 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbddc7e-0bbc-4d3b-9558-f6bdd7189e6b None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:18.986243 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbddc7e-0bbc-4d3b-9558-f6bdd7189e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:18.987036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbddc7e-0bbc-4d3b-9558-f6bdd7189e6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:19.004141 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad953fe-0077-40a6-8b76-6291ce9488a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:19.004790 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad953fe-0077-40a6-8b76-6291ce9488a6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:19.005067 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad953fe-0077-40a6-8b76-6291ce9488a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:19.005335 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad953fe-0077-40a6-8b76-6291ce9488a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:19.006171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad953fe-0077-40a6-8b76-6291ce9488a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:19.006484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad953fe-0077-40a6-8b76-6291ce9488a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:19.007889 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad953fe-0077-40a6-8b76-6291ce9488a6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:19.011382 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbddc7e-0bbc-4d3b-9558-f6bdd7189e6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:19.022708 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 271/560] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:18 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:19.023597 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad953fe-0077-40a6-8b76-6291ce9488a6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:19.024700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-5fbddc7e-0bbc-4d3b-9558-f6bdd7189e6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:19.024700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:19.024700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:19.024700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:19.024700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:19.024700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:19.024700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:19.024700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:19.024700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:19.024700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:19.024700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:19.024700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:19.024700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:19.024700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:19.024700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:19.024700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:19.026611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:19.026611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:19.026611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:19.026611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:19.026611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:19.026611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:19.026611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:19.026611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:19.026611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:19.026611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:19.026611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:19.026611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:19.026611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:19.026611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:19.026611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:19.026611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:19.028373 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:19.028373 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:19.028373 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:19.028373 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:19.028373 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:19.028373 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:19.028373 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:19.028373 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:19.028373 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:19.028373 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:19.028373 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:19.028373 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:19.032378 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 290/561] 10.222.0.121 () {56 vars in 1107 bytes} [Thu Aug 4 10:19:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:19.041999 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbddc7e-0bbc-4d3b-9558-f6bdd7189e6b None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:19.120090 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:19.180211 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad953fe-0077-40a6-8b76-6291ce9488a6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:19.227000 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66010a-9bfa-4e8d-835a-09d9a185b8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:19.228293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66010a-9bfa-4e8d-835a-09d9a185b8d8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:19.229070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66010a-9bfa-4e8d-835a-09d9a185b8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:19.229553 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66010a-9bfa-4e8d-835a-09d9a185b8d8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:19.235510 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbddc7e-0bbc-4d3b-9558-f6bdd7189e6b None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:19.242165 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e66010a-9bfa-4e8d-835a-09d9a185b8d8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:19.244141 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e66010a-9bfa-4e8d-835a-09d9a185b8d8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:19.260866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-5e66010a-9bfa-4e8d-835a-09d9a185b8d8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 10:19:19.260866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:19.260866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:19.260866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:19.260866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:19.260866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:19.260866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:19.260866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:19.260866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:19.260866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:19.260866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:19.260866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:19.260866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:19:19.260866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:19:19.260866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:19:19.260866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:19:19.262075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:19.262075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:19.262075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:19:19.262075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:19.262075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:19:19.262075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:19.262075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:19.262075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:19.262075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:19.262075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:19.262075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:19.262075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:19.262075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:19.262075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:19.262075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:19.262075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:19.262075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:19.262075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:19.263066 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:19.263066 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:19.263066 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:19:19.263066 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:19:19.263066 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:19:19.263066 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:19:19.263066 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:19:19.263066 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:19:19.263066 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 10:19:19.263066 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:19.264414 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:19.267973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 272/562] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:19:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:19.269177 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66010a-9bfa-4e8d-835a-09d9a185b8d8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:19.308302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae51e1b-f8e5-4446-808b-8eac89ca0057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:19.310456 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae51e1b-f8e5-4446-808b-8eac89ca0057 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:19.311124 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae51e1b-f8e5-4446-808b-8eac89ca0057 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:19.311797 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae51e1b-f8e5-4446-808b-8eac89ca0057 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:19.330093 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae51e1b-f8e5-4446-808b-8eac89ca0057 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:19.333787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae51e1b-f8e5-4446-808b-8eac89ca0057 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:19.365846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-fae51e1b-f8e5-4446-808b-8eac89ca0057 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:19.365846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:19.365846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:19.365846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:19.365846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:19.365846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:19.365846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:19.365846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:19.365846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:19.365846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:19.365846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:19.365846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:19.365846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:19:19.365846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:19:19.365846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:19:19.367826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:19:19.367826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:19.367826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:19.367826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:19.367826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:19.367826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:19.367826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:19.367826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:19.367826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:19.367826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:19.367826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:19.367826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:19.367826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:19.367826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:19.367826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:19.367826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:19.367826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:19.369736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:19:19.369736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:19:19.369736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:19:19.369736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:19:19.369736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:19:19.369736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:19:19.369736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:19.369736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:19.372521 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 291/563] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:19.373713 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae51e1b-f8e5-4446-808b-8eac89ca0057 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:19.412690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66010a-9bfa-4e8d-835a-09d9a185b8d8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:19.448980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:19.465414 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d1c93-fc40-4c0e-8da8-6dd678ba0ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:19.466279 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d1c93-fc40-4c0e-8da8-6dd678ba0ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:19.466663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d1c93-fc40-4c0e-8da8-6dd678ba0ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:19.467094 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d1c93-fc40-4c0e-8da8-6dd678ba0ac8 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:19.468005 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d1c93-fc40-4c0e-8da8-6dd678ba0ac8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:19.468402 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d1c93-fc40-4c0e-8da8-6dd678ba0ac8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:19.470090 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d1c93-fc40-4c0e-8da8-6dd678ba0ac8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:19.480323 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 273/564] 10.222.0.121 () {56 vars in 1108 bytes} [Thu Aug 4 10:19:19 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:19.481916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74696490-ec84-425f-8eed-857aa1857802 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:19.482327 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74696490-ec84-425f-8eed-857aa1857802 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:19.482758 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74696490-ec84-425f-8eed-857aa1857802 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:19.484484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 274/565] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:19.485254 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74696490-ec84-425f-8eed-857aa1857802 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:19.521421 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae51e1b-f8e5-4446-808b-8eac89ca0057 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:19.564294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9819b8-a95f-48c5-b2e3-8492dec590ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:19.565138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9819b8-a95f-48c5-b2e3-8492dec590ce None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:19.565490 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9819b8-a95f-48c5-b2e3-8492dec590ce None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:19.565867 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9819b8-a95f-48c5-b2e3-8492dec590ce None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:19.566896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9819b8-a95f-48c5-b2e3-8492dec590ce None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:19.567313 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9819b8-a95f-48c5-b2e3-8492dec590ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:19.569175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9819b8-a95f-48c5-b2e3-8492dec590ce None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:19.603370 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 292/566] 10.222.0.121 () {56 vars in 1142 bytes} [Thu Aug 4 10:19:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:19.603370 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996375f-7f9f-49a2-9dbf-03ce25c5bd4f None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:19.603370 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996375f-7f9f-49a2-9dbf-03ce25c5bd4f None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:19.603370 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996375f-7f9f-49a2-9dbf-03ce25c5bd4f None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:19.605152 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 293/567] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:19 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:19.605987 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2996375f-7f9f-49a2-9dbf-03ce25c5bd4f None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:19.637074 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74696490-ec84-425f-8eed-857aa1857802 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:19.652901 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:19.680439 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f04d6-2a37-46d1-80f5-365a21951b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:19.681087 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f04d6-2a37-46d1-80f5-365a21951b07 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:19.681371 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f04d6-2a37-46d1-80f5-365a21951b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:19.681644 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f04d6-2a37-46d1-80f5-365a21951b07 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:19.682413 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f04d6-2a37-46d1-80f5-365a21951b07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:19.682726 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f04d6-2a37-46d1-80f5-365a21951b07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:19.684216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f04d6-2a37-46d1-80f5-365a21951b07 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:19.697200 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 275/568] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:19:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:19.698512 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85554942-58b4-4f17-a6a0-7423fc235a98 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:19.698811 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85554942-58b4-4f17-a6a0-7423fc235a98 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:19.699065 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85554942-58b4-4f17-a6a0-7423fc235a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:19.725701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-85554942-58b4-4f17-a6a0-7423fc235a98 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:19.727728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-85554942-58b4-4f17-a6a0-7423fc235a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:19.732765 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2996375f-7f9f-49a2-9dbf-03ce25c5bd4f None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:19.771881 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38080284-d6f3-4415-bd4b-e33c0eb38d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:19.772695 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38080284-d6f3-4415-bd4b-e33c0eb38d53 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:19.772967 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38080284-d6f3-4415-bd4b-e33c0eb38d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:19.773236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38080284-d6f3-4415-bd4b-e33c0eb38d53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:19.784148 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38080284-d6f3-4415-bd4b-e33c0eb38d53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:19.785655 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38080284-d6f3-4415-bd4b-e33c0eb38d53 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:19.799006 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:19.801758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-38080284-d6f3-4415-bd4b-e33c0eb38d53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:19.801758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:19.801758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:19.801758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:19.801758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:19.801758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:19.801758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:19.801758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:19.801758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:19.801758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:19.801758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:19.801758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:19.801758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:19.801758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:19.801758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:19.802694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:19.802694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:19.802694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:19.802694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:19.802694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:19.802694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:19.802694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:19.802694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:19.802694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:19.802694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:19.802694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:19.802694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:19.802694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:19.802694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:19.802694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:19.802694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:19.802694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:19.803651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:19.803651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:19.803651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:19.803651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:19.803651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 294/569] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:19.804605 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38080284-d6f3-4415-bd4b-e33c0eb38d53 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:19.852104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 276/570] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:19 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:19:19.852104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14a211-2a05-40b0-8221-5ad46747ad90 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:19.852104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14a211-2a05-40b0-8221-5ad46747ad90 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:19.852104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14a211-2a05-40b0-8221-5ad46747ad90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:19.876122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6e14a211-2a05-40b0-8221-5ad46747ad90 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:19.878208 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6e14a211-2a05-40b0-8221-5ad46747ad90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:19.906843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:19.932318 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38080284-d6f3-4415-bd4b-e33c0eb38d53 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:19.987689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec41d51-5b07-4c4c-acf2-d30d9fac48c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:19.989099 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec41d51-5b07-4c4c-acf2-d30d9fac48c3 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:19.989571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec41d51-5b07-4c4c-acf2-d30d9fac48c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:19.990020 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec41d51-5b07-4c4c-acf2-d30d9fac48c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:19.991512 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 277/571] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:19 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:19:19.993051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14a211-2a05-40b0-8221-5ad46747ad90 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:20.007898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec41d51-5b07-4c4c-acf2-d30d9fac48c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:20.010499 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec41d51-5b07-4c4c-acf2-d30d9fac48c3 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:20.034268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-aec41d51-5b07-4c4c-acf2-d30d9fac48c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:20.034268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:20.034268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:20.034268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:20.034268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:20.034268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:20.034268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:20.034268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:20.034268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:20.034268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:20.034268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:20.034268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:20.034268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:20.034268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:20.034268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:20.041590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:20.041590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:20.041590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:20.041590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:20.041590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:20.041590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:20.041590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:20.041590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:20.041590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:20.041590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:20.041590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:20.041590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:20.041590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:20.041590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:20.041590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:20.041590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:20.041590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:20.043516 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:20.043516 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:20.043516 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:20.043516 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:20.043516 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 295/572] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:20.043516 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec41d51-5b07-4c4c-acf2-d30d9fac48c3 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:20.152020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14a211-2a05-40b0-8221-5ad46747ad90 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:20.162043 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec41d51-5b07-4c4c-acf2-d30d9fac48c3 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:20.165986 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:20.198819 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1e885-f8bb-4d54-80d1-b635fecfbd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:20.198819 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1e885-f8bb-4d54-80d1-b635fecfbd98 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:20.199161 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1e885-f8bb-4d54-80d1-b635fecfbd98 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:20.199425 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1e885-f8bb-4d54-80d1-b635fecfbd98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:20.200186 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff1e885-f8bb-4d54-80d1-b635fecfbd98 None admin] RBAC: Adding query filter params (role.id=2edb704d42834db6acd295fa3a8be5d5, scope.project.id=22768aba760b4307aab131d1312b2bf6, user.id=e16e26784601425c9883acd91f65becc) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:20.200505 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff1e885-f8bb-4d54-80d1-b635fecfbd98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:20.202053 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff1e885-f8bb-4d54-80d1-b635fecfbd98 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:20.208755 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 278/573] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:19:19 2022] GET /identity/v3/role_assignments?user.id=e16e26784601425c9883acd91f65becc&scope.project.id=22768aba760b4307aab131d1312b2bf6&role.id=2edb704d42834db6acd295fa3a8be5d5 => generated 260 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:20.209320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1e885-f8bb-4d54-80d1-b635fecfbd98 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:20.224000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1f148-949a-472e-af67-0e4c9c8a2602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:20.225479 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1f148-949a-472e-af67-0e4c9c8a2602 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:20.226087 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1f148-949a-472e-af67-0e4c9c8a2602 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:20.226658 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1f148-949a-472e-af67-0e4c9c8a2602 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:20.228073 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a1f148-949a-472e-af67-0e4c9c8a2602 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:20.228606 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a1f148-949a-472e-af67-0e4c9c8a2602 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:20.231351 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a1f148-949a-472e-af67-0e4c9c8a2602 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:20.234335 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:20.258715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 296/574] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:20 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:20.261675 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1f148-949a-472e-af67-0e4c9c8a2602 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:20.401247 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1e885-f8bb-4d54-80d1-b635fecfbd98 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:20.448868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6a416-d679-4f23-81d8-92f7ba02a5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:20.449897 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6a416-d679-4f23-81d8-92f7ba02a5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:20.449897 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6a416-d679-4f23-81d8-92f7ba02a5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:20.450151 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6a416-d679-4f23-81d8-92f7ba02a5c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:20.451015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1f148-949a-472e-af67-0e4c9c8a2602 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:20.469438 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6a416-d679-4f23-81d8-92f7ba02a5c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:20.480086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-dcc6a416-d679-4f23-81d8-92f7ba02a5c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:20.480086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:20.480086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:20.480086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:20.480086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:20.480086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:20.480086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:20.480086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:20.480086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:20.480086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:20.480086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:20.480086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:20.480086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:20.480086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:20.480086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:20.480086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:20.481715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:20.481715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:20.481715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:20.481715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:20.481715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:20.481715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:20.481715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:20.481715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:20.481715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:20.481715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:20.481715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:20.481715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:20.481715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:20.481715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:20.481715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:20.481715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:20.483527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:20.483527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:20.483527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:20.483527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:20.483527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:20.483527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:20.483527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:20.483527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:20.483527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:20.483527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:20.483527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:20.483527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:20.483527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 279/575] 10.222.0.121 () {56 vars in 1107 bytes} [Thu Aug 4 10:19:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:20.484897 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6a416-d679-4f23-81d8-92f7ba02a5c4 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:20.524876 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d442ccb-3db1-48b7-9a84-de7ac625336d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:20.526421 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d442ccb-3db1-48b7-9a84-de7ac625336d None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:20.526877 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d442ccb-3db1-48b7-9a84-de7ac625336d None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:20.527332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d442ccb-3db1-48b7-9a84-de7ac625336d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:20.533263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:20.547873 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d442ccb-3db1-48b7-9a84-de7ac625336d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:20.556989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-8d442ccb-3db1-48b7-9a84-de7ac625336d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:20.556989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:20.556989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:20.556989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:20.556989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:20.556989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:20.556989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:20.556989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:20.556989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:20.556989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:20.556989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:20.556989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:20.556989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:20.556989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:20.556989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:20.556989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:20.558907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:20.558907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:20.558907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:20.558907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:20.558907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:20.558907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:20.558907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:20.558907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:20.558907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:20.558907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:20.558907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:20.558907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:20.558907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:20.558907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:20.558907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:20.558907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:20.560687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:20.560687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:20.560687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:20.560687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:20.560687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:20.560687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:20.560687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:20.560687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:20.560687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:20.560687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:20.560687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:20.560687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:20.560687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 297/576] 10.222.0.121 () {56 vars in 1107 bytes} [Thu Aug 4 10:19:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:20.560687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d442ccb-3db1-48b7-9a84-de7ac625336d None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:20.670287 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:20.675916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6a416-d679-4f23-81d8-92f7ba02a5c4 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:20.735661 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a81bd9-74a5-4fe2-9a5f-0ea7b1403412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:20.736384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a81bd9-74a5-4fe2-9a5f-0ea7b1403412 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:20.736790 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a81bd9-74a5-4fe2-9a5f-0ea7b1403412 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:20.737233 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a81bd9-74a5-4fe2-9a5f-0ea7b1403412 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:20.738620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a81bd9-74a5-4fe2-9a5f-0ea7b1403412 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:20.739088 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a81bd9-74a5-4fe2-9a5f-0ea7b1403412 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:20.741574 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a81bd9-74a5-4fe2-9a5f-0ea7b1403412 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:20.753380 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d442ccb-3db1-48b7-9a84-de7ac625336d None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:20.768790 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 280/577] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:20 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:20.768790 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a81bd9-74a5-4fe2-9a5f-0ea7b1403412 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:20.806793 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ec054-6c07-44aa-a40e-76eb0eeeff61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:20.807478 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ec054-6c07-44aa-a40e-76eb0eeeff61 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:20.807758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ec054-6c07-44aa-a40e-76eb0eeeff61 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:20.808075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ec054-6c07-44aa-a40e-76eb0eeeff61 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:20.817431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ec054-6c07-44aa-a40e-76eb0eeeff61 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:20.819070 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ec054-6c07-44aa-a40e-76eb0eeeff61 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:20.832723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-477ec054-6c07-44aa-a40e-76eb0eeeff61 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:19:20.832723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:20.832723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:20.832723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:20.832723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:20.832723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:20.832723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:20.832723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:20.832723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:20.832723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:20.832723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:20.832723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:20.832723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:19:20.832723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:19:20.832723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:19:20.832723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:19:20.834779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:20.834779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:20.834779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:19:20.834779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:20.834779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:19:20.834779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:20.834779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:20.834779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:20.834779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:20.834779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:20.834779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:20.834779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:20.834779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:20.834779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:20.834779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:20.834779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:20.834779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:20.834779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:20.836657 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:20.836657 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:20.836657 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:19:20.836657 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:19:20.836657 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:19:20.836657 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:19:20.836657 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:19:20.836657 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:19:20.836657 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:19:20.836657 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:20.836657 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 298/578] 10.222.0.121 () {56 vars in 1098 bytes} [Thu Aug 4 10:19:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:20.836657 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ec054-6c07-44aa-a40e-76eb0eeeff61 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:20.931583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:20.951362 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a81bd9-74a5-4fe2-9a5f-0ea7b1403412 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:20.967501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ec054-6c07-44aa-a40e-76eb0eeeff61 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:20.983532 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:21.007848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc587a-5a7c-4cef-9bb6-cd3b26d53180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:21.008773 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc587a-5a7c-4cef-9bb6-cd3b26d53180 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:21.008773 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc587a-5a7c-4cef-9bb6-cd3b26d53180 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:21.009065 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc587a-5a7c-4cef-9bb6-cd3b26d53180 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:21.009652 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc587a-5a7c-4cef-9bb6-cd3b26d53180 None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:21.009958 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc587a-5a7c-4cef-9bb6-cd3b26d53180 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:21.011371 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc587a-5a7c-4cef-9bb6-cd3b26d53180 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:21.015193 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97027e-c5b3-4b1d-9d8e-4ae37d748a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:21.016138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97027e-c5b3-4b1d-9d8e-4ae37d748a82 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:21.016606 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97027e-c5b3-4b1d-9d8e-4ae37d748a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:21.017089 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97027e-c5b3-4b1d-9d8e-4ae37d748a82 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:21.018571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97027e-c5b3-4b1d-9d8e-4ae37d748a82 None admin] RBAC: Adding query filter params (name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:21.019419 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97027e-c5b3-4b1d-9d8e-4ae37d748a82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:21.021320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97027e-c5b3-4b1d-9d8e-4ae37d748a82 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:21.021443 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 281/579] 10.222.0.121 () {56 vars in 1108 bytes} [Thu Aug 4 10:19:20 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:21.022756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc587a-5a7c-4cef-9bb6-cd3b26d53180 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:21.042602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 299/580] 10.222.0.121 () {56 vars in 1108 bytes} [Thu Aug 4 10:19:20 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:21.042602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97027e-c5b3-4b1d-9d8e-4ae37d748a82 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:21.185289 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc587a-5a7c-4cef-9bb6-cd3b26d53180 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:21.204943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97027e-c5b3-4b1d-9d8e-4ae37d748a82 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:21.232599 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f0e6e-2043-4479-8641-4cfb767eedeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:21.233288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f0e6e-2043-4479-8641-4cfb767eedeb None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:21.233545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f0e6e-2043-4479-8641-4cfb767eedeb None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:21.233832 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f0e6e-2043-4479-8641-4cfb767eedeb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:21.242398 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26f0e6e-2043-4479-8641-4cfb767eedeb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:21.243866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26f0e6e-2043-4479-8641-4cfb767eedeb None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:21.246834 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac97f8-2c4f-4675-9c30-be712bbb0ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:21.247243 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac97f8-2c4f-4675-9c30-be712bbb0ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:21.247531 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac97f8-2c4f-4675-9c30-be712bbb0ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:21.247810 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac97f8-2c4f-4675-9c30-be712bbb0ee1 None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:21.248681 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac97f8-2c4f-4675-9c30-be712bbb0ee1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:21.248997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac97f8-2c4f-4675-9c30-be712bbb0ee1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:21.250546 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac97f8-2c4f-4675-9c30-be712bbb0ee1 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:21.256524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-f26f0e6e-2043-4479-8641-4cfb767eedeb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:21.256524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:21.256524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:21.256524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:21.256524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:21.256524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:21.256524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:21.256524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:21.256524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:21.256524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:21.256524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:21.256524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:21.256524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:19:21.256524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:19:21.256524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:19:21.258492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:19:21.258492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:21.258492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:21.258492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:21.258492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:21.258492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:21.258492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:21.258492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:21.258492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:21.258492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:21.258492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:21.258492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:21.258492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:21.258492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:21.258492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:21.258492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:21.258492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:21.260406 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:19:21.260406 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:19:21.260406 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:19:21.260406 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:19:21.260406 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:19:21.260406 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:19:21.260406 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:21.260406 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:21.260406 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 282/581] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:21.260406 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f0e6e-2043-4479-8641-4cfb767eedeb None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:21.286192 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 300/582] 10.222.0.121 () {56 vars in 1138 bytes} [Thu Aug 4 10:19:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:21.286192 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac97f8-2c4f-4675-9c30-be712bbb0ee1 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:21.289817 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:21.328284 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:21.426773 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f0e6e-2043-4479-8641-4cfb767eedeb None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:21.443460 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac97f8-2c4f-4675-9c30-be712bbb0ee1 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:21.473897 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c42d3-0f7a-4aa2-95bd-1b8c015454c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:21.475012 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c42d3-0f7a-4aa2-95bd-1b8c015454c5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:21.475622 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c42d3-0f7a-4aa2-95bd-1b8c015454c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:21.476140 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c42d3-0f7a-4aa2-95bd-1b8c015454c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:21.491201 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c42d3-0f7a-4aa2-95bd-1b8c015454c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:21.492672 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba33def-5a3f-4c5c-8fbb-7fc989dccc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:21.493299 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c42d3-0f7a-4aa2-95bd-1b8c015454c5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:21.493751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba33def-5a3f-4c5c-8fbb-7fc989dccc9a None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:21.494266 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba33def-5a3f-4c5c-8fbb-7fc989dccc9a None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:21.494658 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba33def-5a3f-4c5c-8fbb-7fc989dccc9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:21.510583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-457c42d3-0f7a-4aa2-95bd-1b8c015454c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:21.510583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:21.510583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:21.510583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:21.510583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:21.510583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:21.510583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:21.510583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:21.510583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:21.510583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:21.510583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:21.510583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:21.510583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:21.510583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:21.510583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:21.511898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:21.511898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:21.511898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:21.511898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:21.511898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:21.511898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:21.511898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:21.511898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:21.511898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:21.511898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:21.511898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:21.511898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:21.511898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:21.511898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:21.511898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:21.511898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:21.511898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:21.513120 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:21.513120 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:21.513120 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:21.513120 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:21.513120 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 283/583] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:21.513120 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c42d3-0f7a-4aa2-95bd-1b8c015454c5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:21.516916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba33def-5a3f-4c5c-8fbb-7fc989dccc9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:21.518440 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba33def-5a3f-4c5c-8fbb-7fc989dccc9a None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:21.534939 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-5ba33def-5a3f-4c5c-8fbb-7fc989dccc9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:21.534939 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:21.534939 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:21.534939 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:21.534939 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:21.534939 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:21.534939 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:21.534939 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:21.534939 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:21.534939 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:21.534939 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:21.534939 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:21.534939 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:21.534939 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:21.534939 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:21.537173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:21.537173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:21.537173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:21.537173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:21.537173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:21.537173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:21.537173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:21.537173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:21.537173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:21.537173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:21.537173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:21.537173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:21.537173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:21.537173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:21.537173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:21.537173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:21.537173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:21.539025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:21.539025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:21.539025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:21.539025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:21.539025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 301/584] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:21.541968 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba33def-5a3f-4c5c-8fbb-7fc989dccc9a None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:21.609840 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:21.657265 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c42d3-0f7a-4aa2-95bd-1b8c015454c5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:21.667012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:21.686494 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba33def-5a3f-4c5c-8fbb-7fc989dccc9a None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:21.706308 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c848b33-f16e-438e-96c4-a6605e0f8a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:21.707088 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c848b33-f16e-438e-96c4-a6605e0f8a00 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:21.707440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c848b33-f16e-438e-96c4-a6605e0f8a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:21.707779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c848b33-f16e-438e-96c4-a6605e0f8a00 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:21.708646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c848b33-f16e-438e-96c4-a6605e0f8a00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:21.709016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c848b33-f16e-438e-96c4-a6605e0f8a00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:21.710531 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c848b33-f16e-438e-96c4-a6605e0f8a00 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:21.724567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 284/585] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:19:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:21.725595 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c848b33-f16e-438e-96c4-a6605e0f8a00 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:21.747464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edc1e9-018b-4614-a02f-504a248c1353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:21.749059 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edc1e9-018b-4614-a02f-504a248c1353 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:21.749775 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edc1e9-018b-4614-a02f-504a248c1353 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:21.750428 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edc1e9-018b-4614-a02f-504a248c1353 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:21.770540 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46edc1e9-018b-4614-a02f-504a248c1353 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:21.773129 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46edc1e9-018b-4614-a02f-504a248c1353 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:21.800407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-46edc1e9-018b-4614-a02f-504a248c1353 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:21.800407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:21.800407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:21.800407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:21.800407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:21.800407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:21.800407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:21.800407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:21.800407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:21.800407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:21.800407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:21.800407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:21.800407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:21.800407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:21.800407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:21.802392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:21.802392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:21.802392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:21.802392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:21.802392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:21.802392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:21.802392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:21.802392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:21.802392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:21.802392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:21.802392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:21.802392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:21.802392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:21.802392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:21.802392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:21.802392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:21.802392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:21.804376 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:21.804376 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:21.804376 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:21.804376 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:21.804866 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 302/586] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:21.809131 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edc1e9-018b-4614-a02f-504a248c1353 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:21.919040 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c848b33-f16e-438e-96c4-a6605e0f8a00 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:21.967853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:21.985735 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ee7da-e2e5-4af3-b001-a458a065fe48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:21.986491 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ee7da-e2e5-4af3-b001-a458a065fe48 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:21.986848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ee7da-e2e5-4af3-b001-a458a065fe48 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:21.987192 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ee7da-e2e5-4af3-b001-a458a065fe48 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:21.988375 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ee7da-e2e5-4af3-b001-a458a065fe48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:21.988755 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ee7da-e2e5-4af3-b001-a458a065fe48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:21.989391 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edc1e9-018b-4614-a02f-504a248c1353 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:21.990413 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ee7da-e2e5-4af3-b001-a458a065fe48 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:22.012016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 285/587] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:22.012016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ee7da-e2e5-4af3-b001-a458a065fe48 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:22.061252 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1b33a-2db7-4cdd-9db4-c378da835a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:22.062916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1b33a-2db7-4cdd-9db4-c378da835a92 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:22.063391 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1b33a-2db7-4cdd-9db4-c378da835a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:22.063881 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1b33a-2db7-4cdd-9db4-c378da835a92 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:22.065223 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1b33a-2db7-4cdd-9db4-c378da835a92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:22.065763 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1b33a-2db7-4cdd-9db4-c378da835a92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:22.068442 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1b33a-2db7-4cdd-9db4-c378da835a92 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:22.082055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:22.097910 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 303/588] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:22.097910 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1b33a-2db7-4cdd-9db4-c378da835a92 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:22.184843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ee7da-e2e5-4af3-b001-a458a065fe48 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:22.248935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c576678-32bd-408d-9e35-25e85ee481e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:22.250248 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c576678-32bd-408d-9e35-25e85ee481e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:22.250831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c576678-32bd-408d-9e35-25e85ee481e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:22.251400 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c576678-32bd-408d-9e35-25e85ee481e7 None admin] PATH_INFO: `/v3/projects/22768aba760b4307aab131d1312b2bf6/users/1b31c86bc9334d2b8acc35a035f90442/roles/2edb704d42834db6acd295fa3a8be5d5` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:22.279542 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c576678-32bd-408d-9e35-25e85ee481e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=22768aba760b4307aab131d1312b2bf6, user_id=1b31c86bc9334d2b8acc35a035f90442, role_id=2edb704d42834db6acd295fa3a8be5d5)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:22.280413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1b33a-2db7-4cdd-9db4-c378da835a92 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:22.282598 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c576678-32bd-408d-9e35-25e85ee481e7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:22.304997 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 286/589] 10.222.0.121 () {58 vars in 1444 bytes} [Thu Aug 4 10:19:22 2022] PUT /identity/v3/projects/22768aba760b4307aab131d1312b2bf6/users/1b31c86bc9334d2b8acc35a035f90442/roles/2edb704d42834db6acd295fa3a8be5d5 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:19:22.306640 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c576678-32bd-408d-9e35-25e85ee481e7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:22.369619 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444af9a-50d3-477c-92b0-132d3fb1dbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:22.370633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444af9a-50d3-477c-92b0-132d3fb1dbdc None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:22.370633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444af9a-50d3-477c-92b0-132d3fb1dbdc None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:22.370633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444af9a-50d3-477c-92b0-132d3fb1dbdc None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:22.370633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444af9a-50d3-477c-92b0-132d3fb1dbdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:22.370633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444af9a-50d3-477c-92b0-132d3fb1dbdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:22.377507 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444af9a-50d3-477c-92b0-132d3fb1dbdc None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:22.379602 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:22.423892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 304/590] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:22.423892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444af9a-50d3-477c-92b0-132d3fb1dbdc None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:22.518593 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c576678-32bd-408d-9e35-25e85ee481e7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:22.554257 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:22.582113 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760e467-1790-47c3-81a4-868a60195d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:22.582791 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760e467-1790-47c3-81a4-868a60195d75 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:22.583078 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760e467-1790-47c3-81a4-868a60195d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:22.583346 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760e467-1790-47c3-81a4-868a60195d75 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:22.592357 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760e467-1790-47c3-81a4-868a60195d75 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:22.593840 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760e467-1790-47c3-81a4-868a60195d75 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:22.607609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-1760e467-1790-47c3-81a4-868a60195d75 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 10:19:22.607609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:22.607609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:22.607609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:22.607609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:22.607609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:22.607609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:22.607609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:22.607609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:22.607609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:22.607609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:22.607609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:22.607609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:19:22.607609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:19:22.607609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:19:22.607609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:19:22.609723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:22.609723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:22.609723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:19:22.609723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:22.609723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:19:22.609723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:22.609723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:22.609723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:22.609723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:22.609723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:22.609723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:22.609723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:22.609723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:22.609723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:22.609723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:22.609723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:22.609723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:22.609723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:22.611814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:22.611814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:22.611814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:19:22.611814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:19:22.611814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:19:22.611814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:19:22.611814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:19:22.611814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:19:22.611814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 10:19:22.611814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:22.611814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 287/591] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:19:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:22.611814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760e467-1790-47c3-81a4-868a60195d75 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:22.633525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444af9a-50d3-477c-92b0-132d3fb1dbdc None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:22.678751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b2357-38e0-450c-8e40-dcbc4244407a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:22.679460 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b2357-38e0-450c-8e40-dcbc4244407a None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:22.679738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b2357-38e0-450c-8e40-dcbc4244407a None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:22.680052 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b2357-38e0-450c-8e40-dcbc4244407a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:22.689363 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b2357-38e0-450c-8e40-dcbc4244407a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:22.690910 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b2357-38e0-450c-8e40-dcbc4244407a None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:22.704458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-ac2b2357-38e0-450c-8e40-dcbc4244407a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 10:19:22.704458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:22.704458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:22.704458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:22.704458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:22.704458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:22.704458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:22.704458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:22.704458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:22.704458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:22.704458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:22.704458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:22.704458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:19:22.704458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:19:22.704458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:19:22.704458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:19:22.705737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:22.705737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:22.705737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:19:22.705737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:22.705737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:19:22.705737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:22.705737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:22.705737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:22.705737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:22.705737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:22.705737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:22.705737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:22.705737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:22.705737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:22.705737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:22.705737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:22.705737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:22.705737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:22.706726 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:22.706726 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:22.706726 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:19:22.706726 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:19:22.706726 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:19:22.706726 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:19:22.706726 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:19:22.706726 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:19:22.706726 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 10:19:22.706726 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:22.706726 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 305/592] 10.222.0.121 () {56 vars in 1107 bytes} [Thu Aug 4 10:19:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:22.707623 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4382d-e900-40b0-9920-5f78e7abd570 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:22.707905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4382d-e900-40b0-9920-5f78e7abd570 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:22.708205 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4382d-e900-40b0-9920-5f78e7abd570 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:22.709829 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 306/593] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:22.710428 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4382d-e900-40b0-9920-5f78e7abd570 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:22.812263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:22.823146 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760e467-1790-47c3-81a4-868a60195d75 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:22.870866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3eab4-52b9-4ca8-94d7-a4d608964e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:22.872266 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3eab4-52b9-4ca8-94d7-a4d608964e97 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:22.872907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3eab4-52b9-4ca8-94d7-a4d608964e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:22.873551 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3eab4-52b9-4ca8-94d7-a4d608964e97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:22.879831 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4382d-e900-40b0-9920-5f78e7abd570 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:22.888156 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec3eab4-52b9-4ca8-94d7-a4d608964e97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:22.891416 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec3eab4-52b9-4ca8-94d7-a4d608964e97 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:22.901770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:22.911099 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-2ec3eab4-52b9-4ca8-94d7-a4d608964e97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:22.911099 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:22.911099 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:22.911099 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:22.911099 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:22.911099 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:22.911099 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:22.911099 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:22.911099 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:22.911099 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:22.911099 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:22.911099 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:22.911099 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:19:22.911099 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:19:22.911099 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:19:22.913261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:19:22.913261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:22.913261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:22.913261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:22.913261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:22.913261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:22.913261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:22.913261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:22.913261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:22.913261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:22.913261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:22.913261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:22.913261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:22.913261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:22.913261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:22.913261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:22.913261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:22.915367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:19:22.915367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:19:22.915367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:19:22.915367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:19:22.915367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:19:22.915367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:19:22.915367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:22.915367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:22.915367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 288/594] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:22.915367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3eab4-52b9-4ca8-94d7-a4d608964e97 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:22.939848 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeafc129-2bac-4498-b6e1-a56424843476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:22.941762 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafc129-2bac-4498-b6e1-a56424843476 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:22.942441 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafc129-2bac-4498-b6e1-a56424843476 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:22.943578 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafc129-2bac-4498-b6e1-a56424843476 None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:22.945364 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeafc129-2bac-4498-b6e1-a56424843476 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:22.946340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeafc129-2bac-4498-b6e1-a56424843476 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:22.950255 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeafc129-2bac-4498-b6e1-a56424843476 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:22.995997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 307/595] 10.222.0.121 () {56 vars in 1142 bytes} [Thu Aug 4 10:19:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:22.995997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37573341-5d77-4a18-b48e-566885a1ae1a None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:22.995997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37573341-5d77-4a18-b48e-566885a1ae1a None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:22.995997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37573341-5d77-4a18-b48e-566885a1ae1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:23.029096 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-37573341-5d77-4a18-b48e-566885a1ae1a None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:23.031348 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-37573341-5d77-4a18-b48e-566885a1ae1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:23.091137 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3eab4-52b9-4ca8-94d7-a4d608964e97 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:23.136191 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d72e97-1784-4477-ad44-14af7669cb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:23.137022 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d72e97-1784-4477-ad44-14af7669cb0f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:23.137408 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d72e97-1784-4477-ad44-14af7669cb0f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:23.137813 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d72e97-1784-4477-ad44-14af7669cb0f None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:23.138875 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d72e97-1784-4477-ad44-14af7669cb0f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:23.139207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d72e97-1784-4477-ad44-14af7669cb0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:23.140820 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d72e97-1784-4477-ad44-14af7669cb0f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:23.164519 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 308/596] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:22 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:19:23.165283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 289/597] 10.222.0.121 () {56 vars in 1144 bytes} [Thu Aug 4 10:19:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:23.166278 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d72e97-1784-4477-ad44-14af7669cb0f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:23.167203 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37573341-5d77-4a18-b48e-566885a1ae1a None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:23.195924 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:23.298213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37573341-5d77-4a18-b48e-566885a1ae1a None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:23.308776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:23.345001 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ecc5a-151a-46b0-9253-00fa69a0d953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:23.345955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ecc5a-151a-46b0-9253-00fa69a0d953 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:23.346403 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ecc5a-151a-46b0-9253-00fa69a0d953 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:23.346843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ecc5a-151a-46b0-9253-00fa69a0d953 None admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:23.347804 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ecc5a-151a-46b0-9253-00fa69a0d953 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:23.348457 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ecc5a-151a-46b0-9253-00fa69a0d953 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:23.350623 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ecc5a-151a-46b0-9253-00fa69a0d953 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:23.374008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 309/598] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:19:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:23.374008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ecc5a-151a-46b0-9253-00fa69a0d953 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:23.383021 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d72e97-1784-4477-ad44-14af7669cb0f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:23.428065 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25f393-464f-4030-b964-1d48c5a1bcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:23.429467 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25f393-464f-4030-b964-1d48c5a1bcfa None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:23.429912 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25f393-464f-4030-b964-1d48c5a1bcfa None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:23.430356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25f393-464f-4030-b964-1d48c5a1bcfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:23.450197 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25f393-464f-4030-b964-1d48c5a1bcfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:23.453164 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25f393-464f-4030-b964-1d48c5a1bcfa None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:23.482703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-bb25f393-464f-4030-b964-1d48c5a1bcfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:23.482703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:23.482703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:23.482703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:23.482703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:23.482703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:23.482703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:23.482703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:23.482703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:23.482703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:23.482703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:23.482703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:23.482703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:23.482703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:23.482703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:23.483938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:23.483938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:23.483938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:23.483938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:23.483938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:23.483938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:23.483938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:23.483938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:23.483938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:23.483938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:23.483938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:23.483938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:23.483938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:23.483938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:23.483938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:23.483938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:23.483938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:23.485055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:23.485055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:23.485055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:23.485055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:23.488717 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 290/599] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:23.491113 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25f393-464f-4030-b964-1d48c5a1bcfa None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:23.519115 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ecc5a-151a-46b0-9253-00fa69a0d953 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:23.566782 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f1271f-dc0a-49fe-a7ee-a4b4bf965d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:23.567558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1271f-dc0a-49fe-a7ee-a4b4bf965d61 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:23.567838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1271f-dc0a-49fe-a7ee-a4b4bf965d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:23.568181 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1271f-dc0a-49fe-a7ee-a4b4bf965d61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:23.587922 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f1271f-dc0a-49fe-a7ee-a4b4bf965d61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:23.589485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f1271f-dc0a-49fe-a7ee-a4b4bf965d61 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:23.603496 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:23.607204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-b2f1271f-dc0a-49fe-a7ee-a4b4bf965d61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:23.607204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:23.607204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:23.607204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:23.607204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:23.607204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:23.607204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:23.607204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:23.607204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:23.607204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:23.607204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:23.607204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:23.607204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:23.607204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:23.607204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:23.609694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:23.609694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:23.609694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:23.609694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:23.609694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:23.609694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:23.609694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:23.609694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:23.609694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:23.609694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:23.609694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:23.609694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:23.609694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:23.609694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:23.609694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:23.609694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:23.609694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:23.612773 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:23.612773 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:23.612773 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:23.612773 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:23.612773 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 310/600] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:23.612773 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f1271f-dc0a-49fe-a7ee-a4b4bf965d61 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:23.637754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:23.660495 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25f393-464f-4030-b964-1d48c5a1bcfa None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:23.705737 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e5984-bee7-442c-81c7-93fc791506f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:23.706563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e5984-bee7-442c-81c7-93fc791506f0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:23.706787 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e5984-bee7-442c-81c7-93fc791506f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:23.707085 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e5984-bee7-442c-81c7-93fc791506f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:23.726524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5e5984-bee7-442c-81c7-93fc791506f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:23.737238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-8b5e5984-bee7-442c-81c7-93fc791506f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:23.737238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:23.737238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:23.737238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:23.737238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:23.737238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:23.737238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:23.737238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:23.737238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:23.737238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:23.737238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:23.737238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:23.737238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:23.737238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:23.737238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:23.737238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:23.739637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:23.739637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:23.739637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:23.739637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:23.739637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:23.739637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:23.739637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:23.739637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:23.739637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:23.739637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:23.739637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:23.739637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:23.739637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:23.739637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:23.739637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:23.739637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:23.741548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:23.741548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:23.741548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:23.741548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:23.741548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:23.741548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:23.741548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:23.741548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:23.741548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:23.741548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:23.741548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:23.741548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:23.741548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 291/601] 10.222.0.121 () {56 vars in 1107 bytes} [Thu Aug 4 10:19:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:23.743041 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e5984-bee7-442c-81c7-93fc791506f0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:23.778875 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f1271f-dc0a-49fe-a7ee-a4b4bf965d61 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:23.840826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa96167-3028-45eb-bbfe-de590588b2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:23.841759 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa96167-3028-45eb-bbfe-de590588b2e8 None admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:23.842190 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa96167-3028-45eb-bbfe-de590588b2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:23.842659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa96167-3028-45eb-bbfe-de590588b2e8 None admin] PATH_INFO: `/v3/projects/22768aba760b4307aab131d1312b2bf6/users/0d947aeb749546309a8d4b7938087ba0/roles/2edb704d42834db6acd295fa3a8be5d5` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:23.869024 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa96167-3028-45eb-bbfe-de590588b2e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=22768aba760b4307aab131d1312b2bf6, user_id=0d947aeb749546309a8d4b7938087ba0, role_id=2edb704d42834db6acd295fa3a8be5d5)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:23.872199 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa96167-3028-45eb-bbfe-de590588b2e8 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:23.898242 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 311/602] 10.222.0.121 () {58 vars in 1444 bytes} [Thu Aug 4 10:19:23 2022] PUT /identity/v3/projects/22768aba760b4307aab131d1312b2bf6/users/0d947aeb749546309a8d4b7938087ba0/roles/2edb704d42834db6acd295fa3a8be5d5 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:19:23.900219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa96167-3028-45eb-bbfe-de590588b2e8 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:23.952634 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e5984-bee7-442c-81c7-93fc791506f0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:24.003731 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:24.027094 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf101a4-8e1c-4225-8f48-a0b1901d403c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:24.028725 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf101a4-8e1c-4225-8f48-a0b1901d403c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:24.029498 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf101a4-8e1c-4225-8f48-a0b1901d403c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:24.030416 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf101a4-8e1c-4225-8f48-a0b1901d403c None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:24.032143 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf101a4-8e1c-4225-8f48-a0b1901d403c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:24.032982 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf101a4-8e1c-4225-8f48-a0b1901d403c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:24.035658 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf101a4-8e1c-4225-8f48-a0b1901d403c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:24.048537 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:24.065320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 292/603] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:23 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:24.068283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf101a4-8e1c-4225-8f48-a0b1901d403c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:24.127540 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa96167-3028-45eb-bbfe-de590588b2e8 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:24.206383 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71032885-578f-4169-9eed-7ee3adf88474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:24.207590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71032885-578f-4169-9eed-7ee3adf88474 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:24.208119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71032885-578f-4169-9eed-7ee3adf88474 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:24.208604 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71032885-578f-4169-9eed-7ee3adf88474 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:24.210916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71032885-578f-4169-9eed-7ee3adf88474 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:24.211236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71032885-578f-4169-9eed-7ee3adf88474 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:24.214883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71032885-578f-4169-9eed-7ee3adf88474 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:24.239088 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 312/604] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:23 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:24.241265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71032885-578f-4169-9eed-7ee3adf88474 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:24.267716 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf101a4-8e1c-4225-8f48-a0b1901d403c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:24.339506 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346e085-c2f8-44c3-8906-78003423eaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:24.340795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346e085-c2f8-44c3-8906-78003423eaed None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:24.341283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346e085-c2f8-44c3-8906-78003423eaed None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:24.341760 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346e085-c2f8-44c3-8906-78003423eaed None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:24.343225 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346e085-c2f8-44c3-8906-78003423eaed None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:24.343761 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346e085-c2f8-44c3-8906-78003423eaed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:24.346486 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346e085-c2f8-44c3-8906-78003423eaed None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:24.368239 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 293/605] 10.222.0.121 () {56 vars in 1108 bytes} [Thu Aug 4 10:19:24 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:24.370071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346e085-c2f8-44c3-8906-78003423eaed None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:24.454441 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71032885-578f-4169-9eed-7ee3adf88474 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:24.474753 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:24.514986 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:24.532792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137829c-2fad-4a2f-853f-9f4ee96486d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:24.533756 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137829c-2fad-4a2f-853f-9f4ee96486d7 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:24.534210 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137829c-2fad-4a2f-853f-9f4ee96486d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:24.534605 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137829c-2fad-4a2f-853f-9f4ee96486d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:24.550385 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137829c-2fad-4a2f-853f-9f4ee96486d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:24.557527 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137829c-2fad-4a2f-853f-9f4ee96486d7 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:24.578639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-4137829c-2fad-4a2f-853f-9f4ee96486d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:24.578639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:24.578639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:24.578639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:24.578639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:24.578639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:24.578639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:24.578639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:24.578639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:24.578639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:24.578639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:24.578639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:24.578639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:19:24.578639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:19:24.578639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:19:24.579692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:19:24.579692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:24.579692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:24.579692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:24.579692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:24.579692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:24.579692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:24.579692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:24.579692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:24.579692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:24.579692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:24.579692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:24.579692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:24.579692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:24.579692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:24.579692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:24.579692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:24.580753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:19:24.580753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:19:24.580753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:19:24.580753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:19:24.580753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:19:24.580753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:19:24.580753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:24.580753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:24.588263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 313/606] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:24.589471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137829c-2fad-4a2f-853f-9f4ee96486d7 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:24.603190 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346e085-c2f8-44c3-8906-78003423eaed None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:24.663012 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ddfb7-dcc2-40b6-bf35-adff063980bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:24.664771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ddfb7-dcc2-40b6-bf35-adff063980bf None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:24.665216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ddfb7-dcc2-40b6-bf35-adff063980bf None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:24.666227 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ddfb7-dcc2-40b6-bf35-adff063980bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:24.675597 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ddfb7-dcc2-40b6-bf35-adff063980bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:24.677703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ddfb7-dcc2-40b6-bf35-adff063980bf None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:24.693932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-7a6ddfb7-dcc2-40b6-bf35-adff063980bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:24.693932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:24.693932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:24.693932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:24.693932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:24.693932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:24.693932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:24.693932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:24.693932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:24.693932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:24.693932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:24.693932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:24.693932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:19:24.693932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:19:24.693932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:19:24.695117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:19:24.695117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:24.695117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:24.695117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:24.695117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:24.695117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:24.695117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:24.695117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:24.695117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:24.695117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:24.695117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:24.695117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:24.695117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:24.695117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:24.695117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:24.695117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:24.695117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:24.696221 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:19:24.696221 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:19:24.696221 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:19:24.696221 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:19:24.696221 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:19:24.696221 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:19:24.696221 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:24.696221 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:24.698907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 294/607] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:24.700172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ddfb7-dcc2-40b6-bf35-adff063980bf None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:24.775078 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137829c-2fad-4a2f-853f-9f4ee96486d7 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:24.837996 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48eb99-d4b9-45cb-a8fa-9e8952742aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:24.839274 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48eb99-d4b9-45cb-a8fa-9e8952742aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:24.839825 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48eb99-d4b9-45cb-a8fa-9e8952742aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:24.840409 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48eb99-d4b9-45cb-a8fa-9e8952742aa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:24.861512 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48eb99-d4b9-45cb-a8fa-9e8952742aa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:24.862708 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ddfb7-dcc2-40b6-bf35-adff063980bf None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:24.865330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48eb99-d4b9-45cb-a8fa-9e8952742aa2 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:24.873554 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:24.894751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-2d48eb99-d4b9-45cb-a8fa-9e8952742aa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:24.894751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:24.894751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:24.894751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:24.894751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:24.894751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:24.894751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:24.894751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:24.894751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:24.894751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:24.894751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:24.894751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:24.894751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:24.894751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:24.894751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:24.895981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:24.895981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:24.895981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:24.895981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:24.895981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:24.895981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:24.895981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:24.895981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:24.895981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:24.895981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:24.895981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:24.895981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:24.895981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:24.895981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:24.895981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:24.895981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:24.895981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:24.897083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:24.897083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:24.897083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:24.897083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:24.904840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 314/608] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:24.906588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48eb99-d4b9-45cb-a8fa-9e8952742aa2 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:24.914509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c30649-f909-468d-a7f9-9da1d5d92a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:24.915896 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c30649-f909-468d-a7f9-9da1d5d92a45 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:24.916392 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c30649-f909-468d-a7f9-9da1d5d92a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:24.918025 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c30649-f909-468d-a7f9-9da1d5d92a45 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:24.919012 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c30649-f909-468d-a7f9-9da1d5d92a45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:24.919479 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c30649-f909-468d-a7f9-9da1d5d92a45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:24.921244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c30649-f909-468d-a7f9-9da1d5d92a45 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:24.939386 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 295/609] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:19:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:24.940854 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c30649-f909-468d-a7f9-9da1d5d92a45 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:24.988545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:25.084984 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c30649-f909-468d-a7f9-9da1d5d92a45 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:25.088859 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48eb99-d4b9-45cb-a8fa-9e8952742aa2 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:25.130287 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78d628-9920-486d-9ff8-82afdf83af9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:25.130958 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78d628-9920-486d-9ff8-82afdf83af9f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:25.131251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78d628-9920-486d-9ff8-82afdf83af9f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:25.131525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78d628-9920-486d-9ff8-82afdf83af9f None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:25.132275 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78d628-9920-486d-9ff8-82afdf83af9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:25.132586 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78d628-9920-486d-9ff8-82afdf83af9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:25.134164 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78d628-9920-486d-9ff8-82afdf83af9f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:25.150680 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 296/610] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:24 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:25.151569 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78d628-9920-486d-9ff8-82afdf83af9f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:25.156641 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00a1c6-7d0b-4e2f-99b2-184659c4feee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:25.158356 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00a1c6-7d0b-4e2f-99b2-184659c4feee None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:25.158844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00a1c6-7d0b-4e2f-99b2-184659c4feee None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:25.159291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00a1c6-7d0b-4e2f-99b2-184659c4feee None admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:25.160323 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00a1c6-7d0b-4e2f-99b2-184659c4feee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:25.160811 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00a1c6-7d0b-4e2f-99b2-184659c4feee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:25.162587 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00a1c6-7d0b-4e2f-99b2-184659c4feee None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:25.179131 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 315/611] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:19:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:25.180650 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00a1c6-7d0b-4e2f-99b2-184659c4feee None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:25.197550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:25.305302 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78d628-9920-486d-9ff8-82afdf83af9f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:25.347424 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee01a8-88bc-4329-a420-5c117d7cd554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:25.348068 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee01a8-88bc-4329-a420-5c117d7cd554 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:25.348366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee01a8-88bc-4329-a420-5c117d7cd554 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:25.348646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee01a8-88bc-4329-a420-5c117d7cd554 None admin] PATH_INFO: `/v3/projects/22768aba760b4307aab131d1312b2bf6/users/c19bb76d0e2544299916007fbfa62578/roles/2edb704d42834db6acd295fa3a8be5d5` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:25.359175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:25.365585 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00a1c6-7d0b-4e2f-99b2-184659c4feee None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:25.368430 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee01a8-88bc-4329-a420-5c117d7cd554 None admin] RBAC: Authorizing `identity:create_grant(project_id=22768aba760b4307aab131d1312b2bf6, user_id=c19bb76d0e2544299916007fbfa62578, role_id=2edb704d42834db6acd295fa3a8be5d5)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:25.370038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee01a8-88bc-4329-a420-5c117d7cd554 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:25.389308 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 297/612] 10.222.0.121 () {58 vars in 1444 bytes} [Thu Aug 4 10:19:25 2022] PUT /identity/v3/projects/22768aba760b4307aab131d1312b2bf6/users/c19bb76d0e2544299916007fbfa62578/roles/2edb704d42834db6acd295fa3a8be5d5 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:19:25.390665 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee01a8-88bc-4329-a420-5c117d7cd554 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:25.425832 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963241f6-e5b6-416f-a5d0-5675277a4a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:25.426542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963241f6-e5b6-416f-a5d0-5675277a4a4d None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:25.426838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963241f6-e5b6-416f-a5d0-5675277a4a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:25.427124 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963241f6-e5b6-416f-a5d0-5675277a4a4d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:25.437675 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963241f6-e5b6-416f-a5d0-5675277a4a4d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:25.439355 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963241f6-e5b6-416f-a5d0-5675277a4a4d None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:25.455317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-963241f6-e5b6-416f-a5d0-5675277a4a4d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 10:19:25.455317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:25.455317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:25.455317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:25.455317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:25.455317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:25.455317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:25.455317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:25.455317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:25.455317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:25.455317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:25.455317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:25.455317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:19:25.455317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:19:25.455317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:19:25.455317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:19:25.457997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:25.457997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:25.457997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:19:25.457997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:25.457997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:19:25.457997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:25.457997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:25.457997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:25.457997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:25.457997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:25.457997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:25.457997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:25.457997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:25.457997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:25.457997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:25.457997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:25.457997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:25.457997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:25.459989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:25.459989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:25.459989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:19:25.459989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:19:25.459989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:19:25.459989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:19:25.459989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:19:25.459989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:19:25.459989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 10:19:25.459989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:25.459989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 316/613] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:19:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:25.459989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd51b4-a8ae-4be0-9cf6-00c07840de0d None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:25.459989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd51b4-a8ae-4be0-9cf6-00c07840de0d None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:25.461431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd51b4-a8ae-4be0-9cf6-00c07840de0d None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:25.461431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 317/614] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:25.470442 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f51dbc-3897-4029-8b26-f3ec8f7ce327 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:25.470915 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f51dbc-3897-4029-8b26-f3ec8f7ce327 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:25.471548 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f51dbc-3897-4029-8b26-f3ec8f7ce327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:25.504910 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-93f51dbc-3897-4029-8b26-f3ec8f7ce327 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:25.506553 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-93f51dbc-3897-4029-8b26-f3ec8f7ce327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:25.550549 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:25.600617 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee01a8-88bc-4329-a420-5c117d7cd554 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:25.638782 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 318/615] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:25 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:19:25.640534 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f51dbc-3897-4029-8b26-f3ec8f7ce327 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:25.653231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a4f20-a882-443a-9ff4-0743c3c83327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:25.653936 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a4f20-a882-443a-9ff4-0743c3c83327 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:25.654291 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a4f20-a882-443a-9ff4-0743c3c83327 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:25.654578 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a4f20-a882-443a-9ff4-0743c3c83327 None admin] PATH_INFO: `/v3/projects/22768aba760b4307aab131d1312b2bf6/users/90c7eaf4500f4ef496ba72405522006d/roles/2edb704d42834db6acd295fa3a8be5d5` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:25.675206 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4a4f20-a882-443a-9ff4-0743c3c83327 None admin] RBAC: Authorizing `identity:create_grant(project_id=22768aba760b4307aab131d1312b2bf6, user_id=90c7eaf4500f4ef496ba72405522006d, role_id=2edb704d42834db6acd295fa3a8be5d5)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:25.677156 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4a4f20-a882-443a-9ff4-0743c3c83327 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:25.701792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 298/616] 10.222.0.121 () {58 vars in 1444 bytes} [Thu Aug 4 10:19:25 2022] PUT /identity/v3/projects/22768aba760b4307aab131d1312b2bf6/users/90c7eaf4500f4ef496ba72405522006d/roles/2edb704d42834db6acd295fa3a8be5d5 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:19:25.701792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a4f20-a882-443a-9ff4-0743c3c83327 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:25.798571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:25.819760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f51dbc-3897-4029-8b26-f3ec8f7ce327 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:25.849199 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a4f20-a882-443a-9ff4-0743c3c83327 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:25.918769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d794ed-0834-4f70-9db5-98d8503b2fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:25.919989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d794ed-0834-4f70-9db5-98d8503b2fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:25.921061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d794ed-0834-4f70-9db5-98d8503b2fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:25.922613 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d794ed-0834-4f70-9db5-98d8503b2fa1 None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:25.923863 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d794ed-0834-4f70-9db5-98d8503b2fa1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:25.925639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d794ed-0834-4f70-9db5-98d8503b2fa1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:25.927706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d794ed-0834-4f70-9db5-98d8503b2fa1 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:25.930048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:25.961106 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 319/617] 10.222.0.121 () {56 vars in 1140 bytes} [Thu Aug 4 10:19:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:25.968255 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d794ed-0834-4f70-9db5-98d8503b2fa1 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:25.982206 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad813041-77a3-44a0-b190-6afcd19782f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:25.983321 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad813041-77a3-44a0-b190-6afcd19782f0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:25.983786 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad813041-77a3-44a0-b190-6afcd19782f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:25.984315 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad813041-77a3-44a0-b190-6afcd19782f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:26.002504 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad813041-77a3-44a0-b190-6afcd19782f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:26.009345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-ad813041-77a3-44a0-b190-6afcd19782f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:26.009345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:26.009345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:26.009345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:26.009345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:26.009345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:26.009345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:26.009345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:26.009345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:26.009345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:26.009345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:26.009345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:26.009345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:26.009345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:26.009345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:26.009345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:26.010579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:26.010579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:26.010579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:26.010579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:26.010579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:26.010579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:26.010579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:26.010579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:26.010579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:26.010579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:26.010579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:26.010579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:26.010579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:26.010579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:26.010579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:26.010579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:26.011547 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:26.011547 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:26.011547 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:26.011547 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:26.011547 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:26.011547 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:26.011547 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:26.011547 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:26.011547 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:26.011547 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:26.011547 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:26.011547 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:26.014506 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 299/618] 10.222.0.121 () {56 vars in 1107 bytes} [Thu Aug 4 10:19:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:26.025203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad813041-77a3-44a0-b190-6afcd19782f0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:26.160181 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d794ed-0834-4f70-9db5-98d8503b2fa1 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:26.280395 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4fba5-b08b-4b4b-96c9-0bc596449af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:26.281380 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4fba5-b08b-4b4b-96c9-0bc596449af8 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:26.281869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4fba5-b08b-4b4b-96c9-0bc596449af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:26.282326 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4fba5-b08b-4b4b-96c9-0bc596449af8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:26.286454 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad813041-77a3-44a0-b190-6afcd19782f0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:26.304202 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4fba5-b08b-4b4b-96c9-0bc596449af8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:26.306935 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4fba5-b08b-4b4b-96c9-0bc596449af8 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:26.332900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:26.347642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-8df4fba5-b08b-4b4b-96c9-0bc596449af8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:26.347642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:26.347642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:26.347642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:26.347642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:26.347642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:26.347642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:26.347642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:26.347642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:26.347642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:26.347642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:26.347642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:26.347642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:26.347642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:26.347642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:26.349975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:26.349975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:26.349975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:26.349975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:26.349975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:26.349975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:26.349975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:26.349975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:26.349975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:26.349975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:26.349975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:26.349975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:26.349975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:26.349975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:26.349975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:26.349975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:26.349975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:26.352592 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:26.352592 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:26.352592 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:26.352592 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:26.352592 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 320/619] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:19:26.357782 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4fba5-b08b-4b4b-96c9-0bc596449af8 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:26.403184 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff374de-4fab-4661-8f84-0f98237a9226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:26.404898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff374de-4fab-4661-8f84-0f98237a9226 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:26.405682 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff374de-4fab-4661-8f84-0f98237a9226 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:26.406377 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff374de-4fab-4661-8f84-0f98237a9226 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:26.413281 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff374de-4fab-4661-8f84-0f98237a9226 None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:26.413572 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff374de-4fab-4661-8f84-0f98237a9226 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:26.415014 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff374de-4fab-4661-8f84-0f98237a9226 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:26.439084 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 300/620] 10.222.0.121 () {56 vars in 1108 bytes} [Thu Aug 4 10:19:26 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:26.446581 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff374de-4fab-4661-8f84-0f98237a9226 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:26.564475 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:26.631797 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4fba5-b08b-4b4b-96c9-0bc596449af8 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:26.672888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff374de-4fab-4661-8f84-0f98237a9226 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:26.696510 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5539eb-aa3f-4755-a040-0677144973db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:26.697588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5539eb-aa3f-4755-a040-0677144973db None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:26.698180 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5539eb-aa3f-4755-a040-0677144973db None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:26.698685 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5539eb-aa3f-4755-a040-0677144973db None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:26.700063 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5539eb-aa3f-4755-a040-0677144973db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:26.700658 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5539eb-aa3f-4755-a040-0677144973db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:26.702909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5539eb-aa3f-4755-a040-0677144973db None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:26.728161 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 321/621] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:26 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:26.736357 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e837ffe-4a89-44c4-8e14-3ffcd22969e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:26.737256 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e837ffe-4a89-44c4-8e14-3ffcd22969e0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:26.737443 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e837ffe-4a89-44c4-8e14-3ffcd22969e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:26.737604 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5539eb-aa3f-4755-a040-0677144973db None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:26.738141 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e837ffe-4a89-44c4-8e14-3ffcd22969e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:26.765896 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e837ffe-4a89-44c4-8e14-3ffcd22969e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:26.767530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e837ffe-4a89-44c4-8e14-3ffcd22969e0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:26.817594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-1e837ffe-4a89-44c4-8e14-3ffcd22969e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:26.817594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:26.817594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:26.817594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:26.817594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:26.817594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:26.817594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:26.817594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:26.817594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:26.817594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:26.817594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:26.817594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:26.817594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:26.817594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:26.817594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:26.819588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:26.819588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:26.819588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:26.819588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:26.819588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:26.819588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:26.819588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:26.819588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:26.819588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:26.819588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:26.819588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:26.819588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:26.819588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:26.819588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:26.819588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:26.819588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:26.819588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:26.822298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:26.822298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:26.822298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:26.822298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:26.822298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 301/622] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:26.829071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e837ffe-4a89-44c4-8e14-3ffcd22969e0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:26.860054 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:26.934083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5539eb-aa3f-4755-a040-0677144973db None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:26.976585 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34101e-a6cf-4547-b27f-9aa6407a6963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:26.977835 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34101e-a6cf-4547-b27f-9aa6407a6963 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:26.978327 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34101e-a6cf-4547-b27f-9aa6407a6963 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:26.978895 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34101e-a6cf-4547-b27f-9aa6407a6963 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:26.988942 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34101e-a6cf-4547-b27f-9aa6407a6963 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:26.991139 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34101e-a6cf-4547-b27f-9aa6407a6963 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:27.006560 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-0f34101e-a6cf-4547-b27f-9aa6407a6963 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:27.006560 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:27.006560 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:27.006560 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:27.006560 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:27.006560 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:27.006560 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:27.006560 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:27.006560 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:27.006560 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:27.006560 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:27.006560 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:27.006560 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:19:27.006560 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:19:27.006560 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:19:27.008012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:19:27.008012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:27.008012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:27.008012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:27.008012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:27.008012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:27.008012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:27.008012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:27.008012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:27.008012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:27.008012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:27.008012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:27.008012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:27.008012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:27.008012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:27.008012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:27.008012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:27.008960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:19:27.008960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:19:27.008960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:19:27.008960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:19:27.008960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:19:27.008960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:19:27.008960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:27.008960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:27.011846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 322/623] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:27.017402 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2c592-41e0-4fb8-92aa-8f3d24634bcd None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:27.017402 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2c592-41e0-4fb8-92aa-8f3d24634bcd None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:27.017402 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2c592-41e0-4fb8-92aa-8f3d24634bcd None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:27.017402 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 323/624] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:27.020740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2c592-41e0-4fb8-92aa-8f3d24634bcd None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:27.029228 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:27.034226 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e837ffe-4a89-44c4-8e14-3ffcd22969e0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:27.087126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234fece0-5426-4cb3-a904-fa16a633670d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:27.087664 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fece0-5426-4cb3-a904-fa16a633670d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:27.087966 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fece0-5426-4cb3-a904-fa16a633670d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:27.088255 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fece0-5426-4cb3-a904-fa16a633670d None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:27.088973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234fece0-5426-4cb3-a904-fa16a633670d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:27.089289 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234fece0-5426-4cb3-a904-fa16a633670d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:27.091512 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234fece0-5426-4cb3-a904-fa16a633670d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:27.110840 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 302/625] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:26 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:27.112605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977ed84-3593-420e-a92f-575fd08d5a7b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:27.113335 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977ed84-3593-420e-a92f-575fd08d5a7b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:27.114097 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977ed84-3593-420e-a92f-575fd08d5a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:27.137929 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4977ed84-3593-420e-a92f-575fd08d5a7b None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:27.139986 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4977ed84-3593-420e-a92f-575fd08d5a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:27.188087 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2c592-41e0-4fb8-92aa-8f3d24634bcd None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:27.217805 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:27.234657 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29550899-9b24-42f9-876c-91792b59f1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:27.235310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29550899-9b24-42f9-876c-91792b59f1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:27.235601 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29550899-9b24-42f9-876c-91792b59f1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:27.235983 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29550899-9b24-42f9-876c-91792b59f1d4 None admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:27.236813 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29550899-9b24-42f9-876c-91792b59f1d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:27.237128 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29550899-9b24-42f9-876c-91792b59f1d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:27.238958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29550899-9b24-42f9-876c-91792b59f1d4 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:27.260964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 324/626] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:19:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:27.260964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29550899-9b24-42f9-876c-91792b59f1d4 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:27.273820 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 303/627] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:27 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:19:27.275224 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977ed84-3593-420e-a92f-575fd08d5a7b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:27.393775 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:27.405978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977ed84-3593-420e-a92f-575fd08d5a7b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:27.406741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29550899-9b24-42f9-876c-91792b59f1d4 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:27.447693 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbbd28-8f28-4ef4-a1fe-1844c9d6bb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:27.449185 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbbd28-8f28-4ef4-a1fe-1844c9d6bb32 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:27.449682 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbbd28-8f28-4ef4-a1fe-1844c9d6bb32 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:27.450210 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbbd28-8f28-4ef4-a1fe-1844c9d6bb32 None admin] PATH_INFO: `/v3/projects/22768aba760b4307aab131d1312b2bf6/users/e16e26784601425c9883acd91f65becc/roles/2edb704d42834db6acd295fa3a8be5d5` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:27.469456 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b4e816-4482-4f93-a0e0-1b6525fd1ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:27.470705 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4e816-4482-4f93-a0e0-1b6525fd1ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:27.471315 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4e816-4482-4f93-a0e0-1b6525fd1ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:27.471822 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4e816-4482-4f93-a0e0-1b6525fd1ef4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:27.473360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbbd28-8f28-4ef4-a1fe-1844c9d6bb32 None admin] RBAC: Authorizing `identity:create_grant(project_id=22768aba760b4307aab131d1312b2bf6, user_id=e16e26784601425c9883acd91f65becc, role_id=2edb704d42834db6acd295fa3a8be5d5)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:27.475282 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbbd28-8f28-4ef4-a1fe-1844c9d6bb32 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:27.490055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b4e816-4482-4f93-a0e0-1b6525fd1ef4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:27.492406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b4e816-4482-4f93-a0e0-1b6525fd1ef4 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:27.511373 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 304/628] 10.222.0.121 () {58 vars in 1444 bytes} [Thu Aug 4 10:19:27 2022] PUT /identity/v3/projects/22768aba760b4307aab131d1312b2bf6/users/e16e26784601425c9883acd91f65becc/roles/2edb704d42834db6acd295fa3a8be5d5 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:19:27.512854 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbbd28-8f28-4ef4-a1fe-1844c9d6bb32 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:27.516969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-44b4e816-4482-4f93-a0e0-1b6525fd1ef4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 10:19:27.516969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:27.516969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:27.516969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:27.516969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:27.516969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:27.516969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:27.516969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:27.516969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:27.516969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:27.516969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:27.516969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:27.516969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:19:27.516969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:19:27.516969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:19:27.516969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:19:27.518219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:27.518219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:27.518219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:19:27.518219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:27.518219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:19:27.518219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:27.518219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:27.518219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:27.518219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:27.518219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:27.518219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:27.518219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:27.518219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:27.518219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:27.518219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:27.518219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:27.518219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:27.518219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:27.519208 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:27.519208 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:27.519208 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:19:27.519208 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:19:27.519208 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:19:27.519208 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:19:27.519208 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:19:27.519208 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:19:27.519208 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 10:19:27.519208 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:27.527114 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 325/629] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:19:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:27.530268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b4e816-4482-4f93-a0e0-1b6525fd1ef4 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:27.634251 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:27.666961 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbbd28-8f28-4ef4-a1fe-1844c9d6bb32 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:27.706846 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b676f43-6c3a-47f4-91d8-5dddb9b769f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:27.708042 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b676f43-6c3a-47f4-91d8-5dddb9b769f3 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:27.708528 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b676f43-6c3a-47f4-91d8-5dddb9b769f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:27.709033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b676f43-6c3a-47f4-91d8-5dddb9b769f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:27.722235 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b676f43-6c3a-47f4-91d8-5dddb9b769f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:27.728126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-8b676f43-6c3a-47f4-91d8-5dddb9b769f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:27.728126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:27.728126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:27.728126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:27.728126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:27.728126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:27.728126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:27.728126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:27.728126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:27.728126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:27.728126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:27.728126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:27.728126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:27.728126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:27.728126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:27.728126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:27.729219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:27.729219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:27.729219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:27.729219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:27.729219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:27.729219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:27.729219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:27.729219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:27.729219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:27.729219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:27.729219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:27.729219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:27.729219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:27.729219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:27.729219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:27.729219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:27.730093 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:27.730093 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:27.730093 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:27.730093 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:27.730093 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:27.730093 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:27.730093 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:27.730093 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:27.730093 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:27.730093 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:27.730093 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:27.730093 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:27.733572 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 305/630] 10.222.0.121 () {56 vars in 1107 bytes} [Thu Aug 4 10:19:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:27.738979 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:27.740994 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b676f43-6c3a-47f4-91d8-5dddb9b769f3 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:27.762128 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b4e816-4482-4f93-a0e0-1b6525fd1ef4 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:27.843865 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08cba5-aedd-480d-817d-45497857f29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:27.844872 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08cba5-aedd-480d-817d-45497857f29d None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:27.845057 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08cba5-aedd-480d-817d-45497857f29d None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:27.845342 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08cba5-aedd-480d-817d-45497857f29d None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:27.846216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08cba5-aedd-480d-817d-45497857f29d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:27.846987 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08cba5-aedd-480d-817d-45497857f29d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:27.848214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08cba5-aedd-480d-817d-45497857f29d None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:27.874243 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 326/631] 10.222.0.121 () {56 vars in 1142 bytes} [Thu Aug 4 10:19:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:27.885185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08cba5-aedd-480d-817d-45497857f29d None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:27.926822 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b676f43-6c3a-47f4-91d8-5dddb9b769f3 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:27.972285 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a26c09-5fb6-41e3-a5b1-2b277fd02e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:27.973382 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a26c09-5fb6-41e3-a5b1-2b277fd02e06 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:27.973945 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a26c09-5fb6-41e3-a5b1-2b277fd02e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:27.974444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a26c09-5fb6-41e3-a5b1-2b277fd02e06 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:27.975501 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a26c09-5fb6-41e3-a5b1-2b277fd02e06 None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:27.976186 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a26c09-5fb6-41e3-a5b1-2b277fd02e06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:27.977926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a26c09-5fb6-41e3-a5b1-2b277fd02e06 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:27.990102 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 306/632] 10.222.0.121 () {56 vars in 1108 bytes} [Thu Aug 4 10:19:27 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:27.997042 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a26c09-5fb6-41e3-a5b1-2b277fd02e06 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:28.063084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:28.068707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08cba5-aedd-480d-817d-45497857f29d None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:28.109965 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f58a21-bc56-43f3-a5e9-deb96371695c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:28.110225 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f58a21-bc56-43f3-a5e9-deb96371695c None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:28.110351 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f58a21-bc56-43f3-a5e9-deb96371695c None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:28.110646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f58a21-bc56-43f3-a5e9-deb96371695c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:28.160646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:28.217977 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f58a21-bc56-43f3-a5e9-deb96371695c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:28.220380 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f58a21-bc56-43f3-a5e9-deb96371695c None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:28.230690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a26c09-5fb6-41e3-a5b1-2b277fd02e06 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:28.297350 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a512431-957b-4f41-a103-5edf544cb687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:28.299172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a512431-957b-4f41-a103-5edf544cb687 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:28.299562 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a512431-957b-4f41-a103-5edf544cb687 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:28.299871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a512431-957b-4f41-a103-5edf544cb687 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:28.324378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a512431-957b-4f41-a103-5edf544cb687 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:28.328971 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a512431-957b-4f41-a103-5edf544cb687 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:28.372746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-6a512431-957b-4f41-a103-5edf544cb687 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:28.372746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:28.372746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:28.372746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:28.372746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:28.372746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:28.372746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:28.372746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:28.372746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:28.372746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:28.372746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:28.372746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:28.372746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:28.372746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:28.372746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:28.374293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:28.374293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:28.374293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:28.374293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:28.374293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:28.374293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:28.374293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:28.374293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:28.374293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:28.374293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:28.374293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:28.374293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:28.374293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:28.374293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:28.374293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:28.374293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:28.374293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:28.375584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:28.375584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:28.375584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:28.375584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:28.376558 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 307/633] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:28.377570 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba611237-b680-452c-9e5a-f8f32907e271 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:28.378002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba611237-b680-452c-9e5a-f8f32907e271 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:28.378146 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba611237-b680-452c-9e5a-f8f32907e271 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:28.379678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 308/634] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:28.385621 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f6f17-c036-4b50-aad4-58fff34ee061 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:28.385621 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f6f17-c036-4b50-aad4-58fff34ee061 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:28.385621 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f6f17-c036-4b50-aad4-58fff34ee061 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:28.385621 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 309/635] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:28 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:28.385621 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f6f17-c036-4b50-aad4-58fff34ee061 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:28.398527 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:28.404968 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 327/636] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:19:27 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:19:28.408419 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b9faa-31f6-4b50-a6a4-c1ef21bf35b9 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:28.409127 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b9faa-31f6-4b50-a6a4-c1ef21bf35b9 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:28.409416 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b9faa-31f6-4b50-a6a4-c1ef21bf35b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:28.430443 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-243b9faa-31f6-4b50-a6a4-c1ef21bf35b9 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:28.431660 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-243b9faa-31f6-4b50-a6a4-c1ef21bf35b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:28.520706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 328/637] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:28 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:19:28.522134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637e2a4-adec-4c0d-a9f1-3976faa783d4 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:28.522499 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637e2a4-adec-4c0d-a9f1-3976faa783d4 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:28.522850 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637e2a4-adec-4c0d-a9f1-3976faa783d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:28.542096 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-2637e2a4-adec-4c0d-a9f1-3976faa783d4 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:28.543453 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-2637e2a4-adec-4c0d-a9f1-3976faa783d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:28.546002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f6f17-c036-4b50-aad4-58fff34ee061 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:28.571594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:28.603721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adb69f-00e6-42d1-b078-398cab710680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:28.604409 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adb69f-00e6-42d1-b078-398cab710680 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:28.604693 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adb69f-00e6-42d1-b078-398cab710680 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:28.604970 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adb69f-00e6-42d1-b078-398cab710680 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:28.605688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14adb69f-00e6-42d1-b078-398cab710680 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:28.605993 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14adb69f-00e6-42d1-b078-398cab710680 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:28.607434 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14adb69f-00e6-42d1-b078-398cab710680 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:28.621696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 310/638] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:28.622586 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adb69f-00e6-42d1-b078-398cab710680 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:28.642965 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 329/639] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:28 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:19:28.644683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637e2a4-adec-4c0d-a9f1-3976faa783d4 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:28.725079 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:28.752393 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adb69f-00e6-42d1-b078-398cab710680 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:28.776270 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637e2a4-adec-4c0d-a9f1-3976faa783d4 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:28.791145 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863cd6d-9638-421c-af7d-caaa1c82ffb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:28.791813 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863cd6d-9638-421c-af7d-caaa1c82ffb4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:28.792120 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863cd6d-9638-421c-af7d-caaa1c82ffb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:28.792741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863cd6d-9638-421c-af7d-caaa1c82ffb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:28.803076 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f863cd6d-9638-421c-af7d-caaa1c82ffb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:28.804857 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f863cd6d-9638-421c-af7d-caaa1c82ffb4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:28.816377 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324898e-1263-4230-980d-2a2e0e8f46d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:28.817056 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324898e-1263-4230-980d-2a2e0e8f46d9 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:28.817348 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324898e-1263-4230-980d-2a2e0e8f46d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:28.817737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324898e-1263-4230-980d-2a2e0e8f46d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:28.821051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-f863cd6d-9638-421c-af7d-caaa1c82ffb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:28.821051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:28.821051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:28.821051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:28.821051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:28.821051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:28.821051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:28.821051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:28.821051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:28.821051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:28.821051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:28.821051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:28.821051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:28.821051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:28.821051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:28.822116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:28.822116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:28.822116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:28.822116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:28.822116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:28.822116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:28.822116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:28.822116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:28.822116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:28.822116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:28.822116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:28.822116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:28.822116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:28.822116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:28.822116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:28.822116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:28.822116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:28.823297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:28.823297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:28.823297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:28.823297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:28.823297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 311/640] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:28.823297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863cd6d-9638-421c-af7d-caaa1c82ffb4 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:28.836525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324898e-1263-4230-980d-2a2e0e8f46d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:28.841754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-2324898e-1263-4230-980d-2a2e0e8f46d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:28.841754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:28.841754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:28.841754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:28.841754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:28.841754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:28.841754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:28.841754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:28.841754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:28.841754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:28.841754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:28.841754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:28.841754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:28.841754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:28.841754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:28.841754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:28.844236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:28.844236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:28.844236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:28.844236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:28.844236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:28.844236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:28.844236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:28.844236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:28.844236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:28.844236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:28.844236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:28.844236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:28.844236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:28.844236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:28.844236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:28.844236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:28.846162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:28.846162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:28.846162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:28.846162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:28.846162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:28.846162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:28.846162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:28.846162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:28.846162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:28.846162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:28.846162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:28.846162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:28.846162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 330/641] 10.222.0.121 () {56 vars in 1107 bytes} [Thu Aug 4 10:19:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:28.846162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324898e-1263-4230-980d-2a2e0e8f46d9 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:28.864482 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:28.974174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863cd6d-9638-421c-af7d-caaa1c82ffb4 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:29.024535 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324898e-1263-4230-980d-2a2e0e8f46d9 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:29.027776 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4befe518-14d3-4711-bbc2-e775d85a6374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:29.028964 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4befe518-14d3-4711-bbc2-e775d85a6374 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:29.029381 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4befe518-14d3-4711-bbc2-e775d85a6374 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:29.029853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4befe518-14d3-4711-bbc2-e775d85a6374 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:29.040547 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4befe518-14d3-4711-bbc2-e775d85a6374 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:29.042230 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4befe518-14d3-4711-bbc2-e775d85a6374 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:29.051546 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:29.057378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-4befe518-14d3-4711-bbc2-e775d85a6374 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:19:29.057378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:29.057378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:29.057378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:29.057378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:29.057378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:29.057378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:29.057378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:29.057378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:29.057378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:29.057378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:29.057378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:29.057378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:19:29.057378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:19:29.057378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:19:29.057378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:19:29.058679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:29.058679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:29.058679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:19:29.058679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:29.058679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:19:29.058679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:29.058679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:29.058679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:29.058679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:29.058679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:29.058679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:29.058679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:29.058679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:29.058679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:29.058679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:29.058679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:29.058679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:29.058679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:29.059723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:29.059723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:29.059723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:19:29.059723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:19:29.059723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:19:29.059723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:19:29.059723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:19:29.059723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:19:29.059723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:19:29.059723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:29.059723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 312/642] 10.222.0.121 () {56 vars in 1098 bytes} [Thu Aug 4 10:19:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:29.060760 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4befe518-14d3-4711-bbc2-e775d85a6374 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:29.080180 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de421e8a-34e3-4138-bc72-80bcc8fcb811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:29.080790 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de421e8a-34e3-4138-bc72-80bcc8fcb811 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:29.081078 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de421e8a-34e3-4138-bc72-80bcc8fcb811 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:29.081357 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de421e8a-34e3-4138-bc72-80bcc8fcb811 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:29.094230 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de421e8a-34e3-4138-bc72-80bcc8fcb811 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:29.099458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-de421e8a-34e3-4138-bc72-80bcc8fcb811 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:29.099458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:29.099458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:29.099458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:29.099458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:29.099458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:29.099458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:29.099458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:29.099458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:29.099458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:29.099458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:29.099458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:29.099458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:29.099458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:29.099458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:29.099458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:29.100830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:29.100830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:29.100830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:29.100830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:29.100830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:29.100830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:29.100830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:29.100830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:29.100830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:29.100830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:29.100830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:29.100830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:29.100830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:29.100830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:29.100830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:29.100830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:29.107700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:29.107700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:29.107700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:29.107700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:29.107700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:29.107700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:29.107700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:29.107700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:29.107700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:29.107700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:29.107700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:29.107700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:29.107700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 331/643] 10.222.0.121 () {56 vars in 1107 bytes} [Thu Aug 4 10:19:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:29.107700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de421e8a-34e3-4138-bc72-80bcc8fcb811 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:29.186111 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:29.210558 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4befe518-14d3-4711-bbc2-e775d85a6374 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:29.244789 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de421e8a-34e3-4138-bc72-80bcc8fcb811 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:29.274566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92366da-a4e9-4922-86b0-6c0b6abda50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:29.275406 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92366da-a4e9-4922-86b0-6c0b6abda50a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:29.278034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92366da-a4e9-4922-86b0-6c0b6abda50a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:29.278034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92366da-a4e9-4922-86b0-6c0b6abda50a None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:29.278034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92366da-a4e9-4922-86b0-6c0b6abda50a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:29.278034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92366da-a4e9-4922-86b0-6c0b6abda50a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:29.278546 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92366da-a4e9-4922-86b0-6c0b6abda50a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:29.292797 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 313/644] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:29.297249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92366da-a4e9-4922-86b0-6c0b6abda50a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:29.318529 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce3ad9-9e2f-4f7b-a683-304f15e896b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:29.319245 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce3ad9-9e2f-4f7b-a683-304f15e896b4 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:29.319630 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce3ad9-9e2f-4f7b-a683-304f15e896b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:29.319915 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce3ad9-9e2f-4f7b-a683-304f15e896b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:29.320697 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce3ad9-9e2f-4f7b-a683-304f15e896b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:29.321014 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce3ad9-9e2f-4f7b-a683-304f15e896b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:29.322437 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce3ad9-9e2f-4f7b-a683-304f15e896b4 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:29.331632 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 332/645] 10.222.0.121 () {56 vars in 1108 bytes} [Thu Aug 4 10:19:29 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:29.337129 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce3ad9-9e2f-4f7b-a683-304f15e896b4 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:29.383221 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:29.417718 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92366da-a4e9-4922-86b0-6c0b6abda50a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:29.458322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372fc63-65f1-4d92-8e54-a317b045a445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:29.458985 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372fc63-65f1-4d92-8e54-a317b045a445 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:29.459337 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372fc63-65f1-4d92-8e54-a317b045a445 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:29.459613 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372fc63-65f1-4d92-8e54-a317b045a445 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:29.460485 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372fc63-65f1-4d92-8e54-a317b045a445 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:29.460800 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372fc63-65f1-4d92-8e54-a317b045a445 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:29.462258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372fc63-65f1-4d92-8e54-a317b045a445 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:29.467743 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce3ad9-9e2f-4f7b-a683-304f15e896b4 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:29.481169 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 314/646] 10.222.0.121 () {56 vars in 1138 bytes} [Thu Aug 4 10:19:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:29.481833 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372fc63-65f1-4d92-8e54-a317b045a445 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:29.486593 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:29.512414 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2000393-5ac4-457d-aa9a-b1133ce37aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:29.513072 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2000393-5ac4-457d-aa9a-b1133ce37aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:29.513354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2000393-5ac4-457d-aa9a-b1133ce37aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:29.513627 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2000393-5ac4-457d-aa9a-b1133ce37aa6 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:29.514361 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2000393-5ac4-457d-aa9a-b1133ce37aa6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:29.514669 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2000393-5ac4-457d-aa9a-b1133ce37aa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:29.516361 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2000393-5ac4-457d-aa9a-b1133ce37aa6 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:29.525482 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 333/647] 10.222.0.121 () {56 vars in 1108 bytes} [Thu Aug 4 10:19:29 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:29.526375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2000393-5ac4-457d-aa9a-b1133ce37aa6 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:29.602798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372fc63-65f1-4d92-8e54-a317b045a445 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:29.642291 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebd6e1-d885-4e39-95f7-30093922c45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:29.642949 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebd6e1-d885-4e39-95f7-30093922c45d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:29.643226 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebd6e1-d885-4e39-95f7-30093922c45d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:29.643498 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebd6e1-d885-4e39-95f7-30093922c45d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:29.651250 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebd6e1-d885-4e39-95f7-30093922c45d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:29.652797 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebd6e1-d885-4e39-95f7-30093922c45d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:29.653449 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2000393-5ac4-457d-aa9a-b1133ce37aa6 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:29.657399 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:29.669827 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-53ebd6e1-d885-4e39-95f7-30093922c45d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:29.669827 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:29.669827 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:29.669827 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:29.669827 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:29.669827 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:29.669827 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:29.669827 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:29.669827 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:29.669827 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:29.669827 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:29.669827 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:29.669827 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:19:29.669827 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:19:29.669827 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:19:29.671036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:19:29.671036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:29.671036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:29.671036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:29.671036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:29.671036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:29.671036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:29.671036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:29.671036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:29.671036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:29.671036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:29.671036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:29.671036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:29.671036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:29.671036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:29.671036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:29.671036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:29.672171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:19:29.672171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:19:29.672171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:19:29.672171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:19:29.672171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:19:29.672171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:19:29.672171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:29.672171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:29.672901 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 315/648] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:29.673643 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebd6e1-d885-4e39-95f7-30093922c45d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:29.699724 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b051933-776f-4db5-b875-906b677d561d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:29.700803 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b051933-776f-4db5-b875-906b677d561d None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:29.701282 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b051933-776f-4db5-b875-906b677d561d None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:29.701780 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b051933-776f-4db5-b875-906b677d561d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:29.713277 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b051933-776f-4db5-b875-906b677d561d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:29.714762 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b051933-776f-4db5-b875-906b677d561d None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:29.732633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-4b051933-776f-4db5-b875-906b677d561d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:29.732633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:29.732633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:29.732633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:29.732633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:29.732633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:29.732633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:29.732633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:29.732633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:29.732633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:29.732633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:29.732633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:29.732633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:29.732633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:29.732633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:29.733715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:29.733715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:29.733715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:29.733715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:29.733715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:29.733715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:29.733715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:29.733715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:29.733715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:29.733715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:29.733715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:29.733715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:29.733715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:29.733715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:29.733715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:29.733715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:29.733715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:29.734635 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:29.734635 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:29.734635 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:29.734635 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:29.734635 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 334/649] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:29.734934 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b051933-776f-4db5-b875-906b677d561d None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:29.752133 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:29.793831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebd6e1-d885-4e39-95f7-30093922c45d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:29.838055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862f58a-d3cf-42bd-bb11-68556168d6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:29.838590 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862f58a-d3cf-42bd-bb11-68556168d6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:29.838873 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862f58a-d3cf-42bd-bb11-68556168d6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:29.839153 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862f58a-d3cf-42bd-bb11-68556168d6c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:29.849614 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b051933-776f-4db5-b875-906b677d561d None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:29.884887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227685b-bfbf-4837-b72a-d5daea8c46c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:29.885555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227685b-bfbf-4837-b72a-d5daea8c46c9 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:29.885840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227685b-bfbf-4837-b72a-d5daea8c46c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:29.886124 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227685b-bfbf-4837-b72a-d5daea8c46c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:29.897390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227685b-bfbf-4837-b72a-d5daea8c46c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:29.898898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227685b-bfbf-4837-b72a-d5daea8c46c9 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:29.914937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-e227685b-bfbf-4837-b72a-d5daea8c46c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:29.914937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:29.914937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:29.914937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:29.914937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:29.914937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:29.914937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:29.914937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:29.914937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:29.914937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:29.914937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:29.914937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:29.914937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:29.914937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:29.914937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:29.916179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:29.916179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:29.916179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:29.916179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:29.916179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:29.916179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:29.916179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:29.916179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:29.916179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:29.916179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:29.916179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:29.916179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:29.916179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:29.916179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:29.916179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:29.916179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:29.916179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:29.917136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:29.917136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:29.917136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:29.917136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:29.917136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 335/650] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:29.917136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227685b-bfbf-4837-b72a-d5daea8c46c9 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:29.926685 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:29.932824 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9862f58a-d3cf-42bd-bb11-68556168d6c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:29.934337 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9862f58a-d3cf-42bd-bb11-68556168d6c2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:29.996813 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:30.026610 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227685b-bfbf-4837-b72a-d5daea8c46c9 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:30.036725 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 316/651] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:19:29 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:19:30.037645 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862f58a-d3cf-42bd-bb11-68556168d6c2 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:30.066345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20cd0b2-dbe0-49e6-8484-4a4a0caa84e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:30.067017 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cd0b2-dbe0-49e6-8484-4a4a0caa84e1 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:30.067301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cd0b2-dbe0-49e6-8484-4a4a0caa84e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:30.067579 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cd0b2-dbe0-49e6-8484-4a4a0caa84e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:30.068391 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20cd0b2-dbe0-49e6-8484-4a4a0caa84e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:30.068776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20cd0b2-dbe0-49e6-8484-4a4a0caa84e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:30.070258 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20cd0b2-dbe0-49e6-8484-4a4a0caa84e1 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:30.083990 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 336/652] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:19:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:30.085570 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28a142-0c08-464f-86ba-7e75d4cbc37c None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:30.085869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28a142-0c08-464f-86ba-7e75d4cbc37c None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:30.086150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28a142-0c08-464f-86ba-7e75d4cbc37c None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:30.088006 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 337/653] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:30.088476 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28a142-0c08-464f-86ba-7e75d4cbc37c None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:30.141234 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862f58a-d3cf-42bd-bb11-68556168d6c2 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:30.157121 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:30.175990 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6448b-385e-4748-8a76-e85b3c7a6a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:30.176672 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6448b-385e-4748-8a76-e85b3c7a6a66 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:30.176941 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6448b-385e-4748-8a76-e85b3c7a6a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:30.177198 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6448b-385e-4748-8a76-e85b3c7a6a66 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:30.177926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6448b-385e-4748-8a76-e85b3c7a6a66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:30.178227 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6448b-385e-4748-8a76-e85b3c7a6a66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:30.179613 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6448b-385e-4748-8a76-e85b3c7a6a66 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:30.191480 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 317/654] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:30.192354 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6448b-385e-4748-8a76-e85b3c7a6a66 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:30.193909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28a142-0c08-464f-86ba-7e75d4cbc37c None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:30.231405 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:30.234946 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26d9fd-1d34-4cc3-92f7-509aa7fd08ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:30.235603 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26d9fd-1d34-4cc3-92f7-509aa7fd08ee None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:30.235893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26d9fd-1d34-4cc3-92f7-509aa7fd08ee None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:30.236257 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26d9fd-1d34-4cc3-92f7-509aa7fd08ee None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:30.236973 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee26d9fd-1d34-4cc3-92f7-509aa7fd08ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:30.237287 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee26d9fd-1d34-4cc3-92f7-509aa7fd08ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:30.239084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee26d9fd-1d34-4cc3-92f7-509aa7fd08ee None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:30.251222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 338/655] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:30.252224 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26d9fd-1d34-4cc3-92f7-509aa7fd08ee None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:30.315511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6448b-385e-4748-8a76-e85b3c7a6a66 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:30.351636 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1dcd6f-674e-47c0-aa4c-31d5ab3985d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:30.352222 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dcd6f-674e-47c0-aa4c-31d5ab3985d2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:30.352512 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dcd6f-674e-47c0-aa4c-31d5ab3985d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:30.352794 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dcd6f-674e-47c0-aa4c-31d5ab3985d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:30.358570 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26d9fd-1d34-4cc3-92f7-509aa7fd08ee None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:30.364227 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1dcd6f-674e-47c0-aa4c-31d5ab3985d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:30.365740 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1dcd6f-674e-47c0-aa4c-31d5ab3985d2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:30.380520 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-ff1dcd6f-674e-47c0-aa4c-31d5ab3985d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:30.380520 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:30.380520 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:30.380520 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:30.380520 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:30.380520 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:30.380520 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:30.380520 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:30.380520 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:30.380520 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:30.380520 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:30.380520 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:30.380520 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:30.380520 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:30.380520 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:30.381607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:30.381607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:30.381607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:30.381607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:30.381607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:30.381607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:30.381607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:30.381607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:30.381607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:30.381607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:30.381607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:30.381607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:30.381607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:30.381607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:30.381607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:30.381607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:30.381607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:30.383028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:30.383028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:30.383028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:30.383028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:30.383028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 318/656] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:30.383028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c4961-2d7f-4bdd-be62-720be4df13b7 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:30.383446 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c4961-2d7f-4bdd-be62-720be4df13b7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:30.383539 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c4961-2d7f-4bdd-be62-720be4df13b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:30.387052 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:30.397025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf2701-14b8-4638-97b5-3e1f9109d9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:30.398119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf2701-14b8-4638-97b5-3e1f9109d9ac None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:30.398409 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf2701-14b8-4638-97b5-3e1f9109d9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:30.398693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf2701-14b8-4638-97b5-3e1f9109d9ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:30.399511 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf2701-14b8-4638-97b5-3e1f9109d9ac None admin] RBAC: Adding query filter params (role.id=2edb704d42834db6acd295fa3a8be5d5, scope.project.id=22768aba760b4307aab131d1312b2bf6, user.id=1b31c86bc9334d2b8acc35a035f90442) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:30.399860 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf2701-14b8-4638-97b5-3e1f9109d9ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:30.401391 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf2701-14b8-4638-97b5-3e1f9109d9ac None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:30.402509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-407c4961-2d7f-4bdd-be62-720be4df13b7 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:30.403749 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-407c4961-2d7f-4bdd-be62-720be4df13b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:30.407570 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 339/657] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:19:30 2022] GET /identity/v3/role_assignments?user.id=1b31c86bc9334d2b8acc35a035f90442&scope.project.id=22768aba760b4307aab131d1312b2bf6&role.id=2edb704d42834db6acd295fa3a8be5d5 => generated 610 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:30.408531 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf2701-14b8-4638-97b5-3e1f9109d9ac None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:30.487090 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 319/658] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:30 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:19:30.488463 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c4961-2d7f-4bdd-be62-720be4df13b7 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:30.501471 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:30.541699 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf2701-14b8-4638-97b5-3e1f9109d9ac None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:30.590485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10583c-3707-4e04-a41a-377ef419212c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:30.591333 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10583c-3707-4e04-a41a-377ef419212c None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:30.591561 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10583c-3707-4e04-a41a-377ef419212c None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:30.591840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10583c-3707-4e04-a41a-377ef419212c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:30.601174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac10583c-3707-4e04-a41a-377ef419212c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:30.602815 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac10583c-3707-4e04-a41a-377ef419212c None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:30.617770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-ac10583c-3707-4e04-a41a-377ef419212c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 10:19:30.617770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:30.617770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:30.617770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:30.617770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:30.617770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:30.617770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:30.617770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:30.617770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:30.617770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:30.617770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:30.617770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:30.617770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:19:30.617770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:19:30.617770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:19:30.617770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:19:30.618855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:30.618855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:30.618855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:19:30.618855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:30.618855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:19:30.618855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:30.618855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:30.618855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:30.618855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:30.618855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:30.618855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:30.618855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:30.618855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:30.618855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:30.618855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:30.618855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:30.618855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:30.618855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:30.619837 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:30.619837 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:30.619837 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:19:30.619837 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:19:30.619837 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:19:30.619837 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:19:30.619837 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:19:30.619837 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:19:30.619837 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 10:19:30.619837 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:30.619837 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 340/659] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:19:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:30.619837 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10583c-3707-4e04-a41a-377ef419212c None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:30.624509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c4961-2d7f-4bdd-be62-720be4df13b7 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:30.669658 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be76ae3-23ad-4abd-b535-19f6e651e409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:30.670698 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be76ae3-23ad-4abd-b535-19f6e651e409 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:30.671040 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be76ae3-23ad-4abd-b535-19f6e651e409 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:30.671272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be76ae3-23ad-4abd-b535-19f6e651e409 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:30.679770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:30.680886 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be76ae3-23ad-4abd-b535-19f6e651e409 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:30.682398 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be76ae3-23ad-4abd-b535-19f6e651e409 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:30.696594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-1be76ae3-23ad-4abd-b535-19f6e651e409 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 10:19:30.696594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:30.696594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:30.696594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:30.696594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:30.696594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:30.696594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:30.696594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:30.696594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:30.696594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:30.696594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:30.696594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:30.696594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:19:30.696594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:19:30.696594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:19:30.696594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:19:30.697695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:30.697695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:30.697695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:19:30.697695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:30.697695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:19:30.697695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:30.697695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:30.697695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:30.697695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:30.697695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:30.697695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:30.697695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:30.697695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:30.697695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:30.697695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:30.697695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:30.697695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:30.697695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:30.698683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:30.698683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:30.698683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:19:30.698683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:19:30.698683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:19:30.698683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:19:30.698683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:19:30.698683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:19:30.698683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 10:19:30.698683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:30.698683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 320/660] 10.222.0.121 () {56 vars in 1107 bytes} [Thu Aug 4 10:19:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:30.698683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be76ae3-23ad-4abd-b535-19f6e651e409 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:30.743860 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10583c-3707-4e04-a41a-377ef419212c None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:30.773434 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:30.779170 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139e337-570e-4b54-a9f0-86ffdce4b449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:30.779827 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139e337-570e-4b54-a9f0-86ffdce4b449 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:30.780150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139e337-570e-4b54-a9f0-86ffdce4b449 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:30.780430 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139e337-570e-4b54-a9f0-86ffdce4b449 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:30.781149 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139e337-570e-4b54-a9f0-86ffdce4b449 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:30.781469 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139e337-570e-4b54-a9f0-86ffdce4b449 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:30.782906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139e337-570e-4b54-a9f0-86ffdce4b449 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:30.796319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 341/661] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:30.797564 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139e337-570e-4b54-a9f0-86ffdce4b449 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:30.817189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be76ae3-23ad-4abd-b535-19f6e651e409 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:30.854437 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac30cf4-4420-408f-a7d0-9c7ba326bcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:30.855444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac30cf4-4420-408f-a7d0-9c7ba326bcad None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:30.855884 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac30cf4-4420-408f-a7d0-9c7ba326bcad None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:30.856338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac30cf4-4420-408f-a7d0-9c7ba326bcad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:30.868880 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac30cf4-4420-408f-a7d0-9c7ba326bcad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:30.874935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-cac30cf4-4420-408f-a7d0-9c7ba326bcad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:30.874935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:30.874935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:30.874935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:30.874935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:30.874935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:30.874935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:30.874935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:30.874935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:30.874935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:30.874935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:30.874935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:30.874935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:30.874935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:30.874935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:30.874935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:30.876168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:30.876168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:30.876168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:30.876168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:30.876168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:30.876168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:30.876168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:30.876168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:30.876168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:30.876168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:30.876168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:30.876168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:30.876168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:30.876168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:30.876168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:30.876168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:30.877244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:30.877244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:30.877244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:30.877244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:30.877244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:30.877244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:30.877244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:30.877244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:30.877244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:30.877244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:30.877244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:30.877244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:30.879989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 321/662] 10.222.0.121 () {56 vars in 1107 bytes} [Thu Aug 4 10:19:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:30.881061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac30cf4-4420-408f-a7d0-9c7ba326bcad None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:30.911015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139e337-570e-4b54-a9f0-86ffdce4b449 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:30.923774 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:30.950277 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b95d7-99a6-4919-8f53-e3068acb8ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:30.950937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b95d7-99a6-4919-8f53-e3068acb8ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:30.951232 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b95d7-99a6-4919-8f53-e3068acb8ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:30.951873 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b95d7-99a6-4919-8f53-e3068acb8ca5 None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:30.952740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b95d7-99a6-4919-8f53-e3068acb8ca5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:30.953057 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b95d7-99a6-4919-8f53-e3068acb8ca5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:30.954686 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b95d7-99a6-4919-8f53-e3068acb8ca5 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:30.973489 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 342/663] 10.222.0.121 () {56 vars in 1142 bytes} [Thu Aug 4 10:19:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:30.974381 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b95d7-99a6-4919-8f53-e3068acb8ca5 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:31.006206 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac30cf4-4420-408f-a7d0-9c7ba326bcad None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:31.036325 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:31.047471 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca67f3-378f-4010-ae9e-678ddf74d96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:31.048216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca67f3-378f-4010-ae9e-678ddf74d96c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:31.048413 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca67f3-378f-4010-ae9e-678ddf74d96c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:31.048696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca67f3-378f-4010-ae9e-678ddf74d96c None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:31.050007 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca67f3-378f-4010-ae9e-678ddf74d96c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:31.050007 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca67f3-378f-4010-ae9e-678ddf74d96c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:31.051413 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca67f3-378f-4010-ae9e-678ddf74d96c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:31.078083 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 322/664] 10.222.0.121 () {56 vars in 1144 bytes} [Thu Aug 4 10:19:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:31.079010 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca67f3-378f-4010-ae9e-678ddf74d96c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:31.088867 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b95d7-99a6-4919-8f53-e3068acb8ca5 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:31.129686 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13ecf6-f3f8-4043-96aa-1734fa0713fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:31.130249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13ecf6-f3f8-4043-96aa-1734fa0713fc None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:31.130529 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13ecf6-f3f8-4043-96aa-1734fa0713fc None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:31.130830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13ecf6-f3f8-4043-96aa-1734fa0713fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:31.138424 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e13ecf6-f3f8-4043-96aa-1734fa0713fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:31.140153 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e13ecf6-f3f8-4043-96aa-1734fa0713fc None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:31.153501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-0e13ecf6-f3f8-4043-96aa-1734fa0713fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:31.153501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:31.153501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:31.153501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:31.153501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:31.153501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:31.153501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:31.153501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:31.153501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:31.153501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:31.153501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:31.153501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:31.153501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:19:31.153501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:19:31.153501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:19:31.154568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:19:31.154568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:31.154568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:31.154568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:31.154568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:31.154568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:31.154568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:31.154568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:31.154568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:31.154568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:31.154568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:31.154568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:31.154568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:31.154568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:31.154568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:31.154568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:31.154568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:31.155523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:19:31.155523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:19:31.155523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:19:31.155523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:19:31.155523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:19:31.155523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:19:31.155523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:31.155523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:31.157701 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 343/665] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:31.158780 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13ecf6-f3f8-4043-96aa-1734fa0713fc None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:31.191791 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:31.201575 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca67f3-378f-4010-ae9e-678ddf74d96c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:31.243556 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641da671-c1b9-4fbd-82cc-e8f988a0e608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:31.244419 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641da671-c1b9-4fbd-82cc-e8f988a0e608 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:31.244699 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641da671-c1b9-4fbd-82cc-e8f988a0e608 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:31.244985 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641da671-c1b9-4fbd-82cc-e8f988a0e608 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:31.245785 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641da671-c1b9-4fbd-82cc-e8f988a0e608 None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:31.246134 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641da671-c1b9-4fbd-82cc-e8f988a0e608 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:31.247550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641da671-c1b9-4fbd-82cc-e8f988a0e608 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:31.257208 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 323/666] 10.222.0.121 () {56 vars in 1108 bytes} [Thu Aug 4 10:19:31 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:31.258159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641da671-c1b9-4fbd-82cc-e8f988a0e608 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:31.294907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13ecf6-f3f8-4043-96aa-1734fa0713fc None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:31.312997 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:31.336187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee19fb-8a42-48b8-b672-c7dfb6dd09b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:31.336836 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee19fb-8a42-48b8-b672-c7dfb6dd09b9 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:31.337124 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee19fb-8a42-48b8-b672-c7dfb6dd09b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:31.337404 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee19fb-8a42-48b8-b672-c7dfb6dd09b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:31.376803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641da671-c1b9-4fbd-82cc-e8f988a0e608 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:31.416191 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3eb0c5-3d78-40f7-b9ec-13c125ac7bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:31.416779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3eb0c5-3d78-40f7-b9ec-13c125ac7bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:31.417068 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3eb0c5-3d78-40f7-b9ec-13c125ac7bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:31.417399 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3eb0c5-3d78-40f7-b9ec-13c125ac7bc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:31.422470 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee19fb-8a42-48b8-b672-c7dfb6dd09b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:31.424034 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee19fb-8a42-48b8-b672-c7dfb6dd09b9 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:31.458023 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:31.504633 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3eb0c5-3d78-40f7-b9ec-13c125ac7bc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:31.506385 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3eb0c5-3d78-40f7-b9ec-13c125ac7bc0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:31.549219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 344/667] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:19:31 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:19:31.550546 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee19fb-8a42-48b8-b672-c7dfb6dd09b9 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:31.555652 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:31.619669 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 324/668] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:19:31 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:19:31.620606 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3eb0c5-3d78-40f7-b9ec-13c125ac7bc0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:31.687772 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee19fb-8a42-48b8-b672-c7dfb6dd09b9 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:31.718568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:31.728874 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7e545-62fb-46fd-b0a9-50e7383f9200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:31.729526 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7e545-62fb-46fd-b0a9-50e7383f9200 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:31.729808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7e545-62fb-46fd-b0a9-50e7383f9200 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:31.730087 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7e545-62fb-46fd-b0a9-50e7383f9200 None admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:31.730872 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca7e545-62fb-46fd-b0a9-50e7383f9200 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:31.731182 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca7e545-62fb-46fd-b0a9-50e7383f9200 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:31.732690 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca7e545-62fb-46fd-b0a9-50e7383f9200 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:31.743457 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3eb0c5-3d78-40f7-b9ec-13c125ac7bc0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:31.750756 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 345/669] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:19:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:31.750756 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7e545-62fb-46fd-b0a9-50e7383f9200 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:31.792869 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c1951-b2c3-40af-b93f-315803fad720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:31.793540 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c1951-b2c3-40af-b93f-315803fad720 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:31.793823 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c1951-b2c3-40af-b93f-315803fad720 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:31.794102 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c1951-b2c3-40af-b93f-315803fad720 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:31.809720 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071c1951-b2c3-40af-b93f-315803fad720 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:31.812209 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071c1951-b2c3-40af-b93f-315803fad720 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:31.822522 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:31.834913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-071c1951-b2c3-40af-b93f-315803fad720 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:31.834913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:31.834913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:31.834913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:31.834913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:31.834913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:31.834913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:31.834913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:31.834913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:31.834913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:31.834913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:31.834913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:31.834913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:31.834913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:31.834913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:31.836167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:31.836167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:31.836167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:31.836167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:31.836167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:31.836167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:31.836167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:31.836167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:31.836167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:31.836167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:31.836167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:31.836167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:31.836167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:31.836167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:31.836167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:31.836167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:31.836167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:31.837270 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:31.837270 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:31.837270 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:31.837270 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:31.838908 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 325/670] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:31.839883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c1951-b2c3-40af-b93f-315803fad720 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:31.874988 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7e545-62fb-46fd-b0a9-50e7383f9200 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:31.927301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9293823-b6ba-49ff-82ea-5e6c9f564836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:31.928137 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9293823-b6ba-49ff-82ea-5e6c9f564836 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:31.928432 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9293823-b6ba-49ff-82ea-5e6c9f564836 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:31.928708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9293823-b6ba-49ff-82ea-5e6c9f564836 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:31.941841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9293823-b6ba-49ff-82ea-5e6c9f564836 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:31.943245 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9293823-b6ba-49ff-82ea-5e6c9f564836 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:31.960794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-e9293823-b6ba-49ff-82ea-5e6c9f564836 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:31.960794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:31.960794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:31.960794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:31.960794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:31.960794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:31.960794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:31.960794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:31.960794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:31.960794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:31.960794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:31.960794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:31.960794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:31.960794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:31.960794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:31.961893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:31.961893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:31.961893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:31.961893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:31.961893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:31.961893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:31.961893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:31.961893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:31.961893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:31.961893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:31.961893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:31.961893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:31.961893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:31.961893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:31.961893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:31.961893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:31.961893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:31.962819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:31.962819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:31.962819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:31.962819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:31.962819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 346/671] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:31.963106 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9293823-b6ba-49ff-82ea-5e6c9f564836 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:31.976006 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c1951-b2c3-40af-b93f-315803fad720 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:32.021080 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:32.025063 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda46fba-af57-482e-8c04-5c33314cfe2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:32.025776 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda46fba-af57-482e-8c04-5c33314cfe2a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:32.026060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda46fba-af57-482e-8c04-5c33314cfe2a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:32.026516 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda46fba-af57-482e-8c04-5c33314cfe2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:32.042190 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda46fba-af57-482e-8c04-5c33314cfe2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:32.043908 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda46fba-af57-482e-8c04-5c33314cfe2a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:32.063925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-bda46fba-af57-482e-8c04-5c33314cfe2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:32.063925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:32.063925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:32.063925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:32.063925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:32.063925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:32.063925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:32.063925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:32.063925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:32.063925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:32.063925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:32.063925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:32.063925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:32.063925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:32.063925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:32.066451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:32.066451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:32.066451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:32.066451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:32.066451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:32.066451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:32.066451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:32.066451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:32.066451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:32.066451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:32.066451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:32.066451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:32.066451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:32.066451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:32.066451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:32.066451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:32.066451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:32.069297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:32.069297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:32.069297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:32.069297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:32.069297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 326/672] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:32.069297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda46fba-af57-482e-8c04-5c33314cfe2a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:32.119752 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9293823-b6ba-49ff-82ea-5e6c9f564836 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:32.149901 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:32.173473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47670a-9d64-4c69-93d7-4f15b272bc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:32.174468 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47670a-9d64-4c69-93d7-4f15b272bc6f None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:32.174933 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47670a-9d64-4c69-93d7-4f15b272bc6f None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:32.175371 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47670a-9d64-4c69-93d7-4f15b272bc6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:32.176754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd47670a-9d64-4c69-93d7-4f15b272bc6f None admin] RBAC: Adding query filter params (role.id=2edb704d42834db6acd295fa3a8be5d5, scope.project.id=22768aba760b4307aab131d1312b2bf6, user.id=0d947aeb749546309a8d4b7938087ba0) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:32.177258 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd47670a-9d64-4c69-93d7-4f15b272bc6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:32.179660 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd47670a-9d64-4c69-93d7-4f15b272bc6f None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:32.189679 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 347/673] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:19:31 2022] GET /identity/v3/role_assignments?user.id=0d947aeb749546309a8d4b7938087ba0&scope.project.id=22768aba760b4307aab131d1312b2bf6&role.id=2edb704d42834db6acd295fa3a8be5d5 => generated 610 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:32.190486 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47670a-9d64-4c69-93d7-4f15b272bc6f None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:32.238631 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda46fba-af57-482e-8c04-5c33314cfe2a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:32.296681 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7d633-752a-42fd-b3d7-ac7cf6633da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:32.299024 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7d633-752a-42fd-b3d7-ac7cf6633da7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:32.299724 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7d633-752a-42fd-b3d7-ac7cf6633da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:32.300244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7d633-752a-42fd-b3d7-ac7cf6633da7 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:32.301234 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7d633-752a-42fd-b3d7-ac7cf6633da7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:32.302177 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7d633-752a-42fd-b3d7-ac7cf6633da7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:32.304904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7d633-752a-42fd-b3d7-ac7cf6633da7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:32.328024 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 327/674] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:32 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:32.328024 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7d633-752a-42fd-b3d7-ac7cf6633da7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:32.367284 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47670a-9d64-4c69-93d7-4f15b272bc6f None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:32.372004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:32.438853 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90159a-cffe-4759-8d81-7a252700e012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:32.439519 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90159a-cffe-4759-8d81-7a252700e012 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:32.439809 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90159a-cffe-4759-8d81-7a252700e012 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:32.440748 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90159a-cffe-4759-8d81-7a252700e012 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:32.440866 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d90159a-cffe-4759-8d81-7a252700e012 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:32.441985 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d90159a-cffe-4759-8d81-7a252700e012 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:32.442559 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d90159a-cffe-4759-8d81-7a252700e012 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:32.462371 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 348/675] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:32 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:32.463782 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90159a-cffe-4759-8d81-7a252700e012 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:32.485177 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7d633-752a-42fd-b3d7-ac7cf6633da7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:32.506964 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:32.543625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ff91f-a815-4735-b46b-aa2eed1fa842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:32.544463 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ff91f-a815-4735-b46b-aa2eed1fa842 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:32.544758 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ff91f-a815-4735-b46b-aa2eed1fa842 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:32.545036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ff91f-a815-4735-b46b-aa2eed1fa842 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:32.545898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ff91f-a815-4735-b46b-aa2eed1fa842 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:32.546270 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ff91f-a815-4735-b46b-aa2eed1fa842 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:32.547721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ff91f-a815-4735-b46b-aa2eed1fa842 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:32.571400 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 328/676] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:32 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:32.575060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ff91f-a815-4735-b46b-aa2eed1fa842 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:32.617112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90159a-cffe-4759-8d81-7a252700e012 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:32.656178 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3cd54-8692-4ae2-9979-8b719f3c1cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:32.656950 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3cd54-8692-4ae2-9979-8b719f3c1cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:32.657231 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3cd54-8692-4ae2-9979-8b719f3c1cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:32.657596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3cd54-8692-4ae2-9979-8b719f3c1cf6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:32.666511 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3cd54-8692-4ae2-9979-8b719f3c1cf6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:32.668186 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3cd54-8692-4ae2-9979-8b719f3c1cf6 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:32.681919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-c9b3cd54-8692-4ae2-9979-8b719f3c1cf6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 10:19:32.681919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:32.681919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:32.681919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:32.681919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:32.681919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:32.681919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:32.681919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:32.681919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:32.681919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:32.681919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:32.681919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:32.681919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:19:32.681919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:19:32.681919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:19:32.681919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:19:32.683002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:32.683002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:32.683002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:19:32.683002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:32.683002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:19:32.683002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:32.683002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:32.683002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:32.683002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:32.683002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:32.683002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:32.683002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:32.683002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:32.683002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:32.683002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:32.683002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:32.683002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:32.683002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:32.684000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:32.684000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:32.684000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:19:32.684000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:19:32.684000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:19:32.684000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:19:32.684000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:19:32.684000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:19:32.684000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 10:19:32.684000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:32.684000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 349/677] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:19:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:32.684648 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3cd54-8692-4ae2-9979-8b719f3c1cf6 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:32.696127 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ff91f-a815-4735-b46b-aa2eed1fa842 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:32.713298 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:32.735318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02922ed7-fed5-40a8-8565-070950a5d100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:32.736187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02922ed7-fed5-40a8-8565-070950a5d100 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:32.736879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02922ed7-fed5-40a8-8565-070950a5d100 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:32.737192 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02922ed7-fed5-40a8-8565-070950a5d100 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:32.745219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02922ed7-fed5-40a8-8565-070950a5d100 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:32.746788 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02922ed7-fed5-40a8-8565-070950a5d100 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:32.759104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-02922ed7-fed5-40a8-8565-070950a5d100 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:32.759104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:32.759104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:32.759104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:32.759104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:32.759104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:32.759104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:32.759104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:32.759104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:32.759104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:32.759104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:32.759104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:32.759104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:19:32.759104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:19:32.759104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:19:32.760142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:19:32.760142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:32.760142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:32.760142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:32.760142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:32.760142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:32.760142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:32.760142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:32.760142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:32.760142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:32.760142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:32.760142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:32.760142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:32.760142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:32.760142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:32.760142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:32.760142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:32.761074 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:19:32.761074 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:19:32.761074 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:19:32.761074 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:19:32.761074 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:19:32.761074 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:19:32.761074 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:32.761074 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:32.761074 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 329/678] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:32.762202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fcb76-6364-4375-9e2f-61ab9cdc7586 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:32.762451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fcb76-6364-4375-9e2f-61ab9cdc7586 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:32.762730 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fcb76-6364-4375-9e2f-61ab9cdc7586 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:32.769169 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 330/679] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:32.769169 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fcb76-6364-4375-9e2f-61ab9cdc7586 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:32.803192 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3cd54-8692-4ae2-9979-8b719f3c1cf6 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:32.808181 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:32.845665 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e509ac9-102e-4856-9e10-f39478a6bcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:32.846900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e509ac9-102e-4856-9e10-f39478a6bcd4 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:32.847336 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e509ac9-102e-4856-9e10-f39478a6bcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:32.847756 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e509ac9-102e-4856-9e10-f39478a6bcd4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:32.857954 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e509ac9-102e-4856-9e10-f39478a6bcd4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:32.859858 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e509ac9-102e-4856-9e10-f39478a6bcd4 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:32.873332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-3e509ac9-102e-4856-9e10-f39478a6bcd4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:32.873332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:32.873332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:32.873332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:32.873332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:32.873332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:32.873332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:32.873332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:32.873332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:32.873332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:32.873332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:32.873332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:32.873332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:19:32.873332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:19:32.873332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:19:32.874471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:19:32.874471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:32.874471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:32.874471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:32.874471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:32.874471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:32.874471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:32.874471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:32.874471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:32.874471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:32.874471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:32.874471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:32.874471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:32.874471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:32.874471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:32.874471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:32.874471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:32.875581 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:19:32.875581 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:19:32.875581 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:19:32.875581 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:19:32.875581 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:19:32.875581 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:19:32.875581 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:32.875581 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:32.877616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 350/680] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:32.878717 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e509ac9-102e-4856-9e10-f39478a6bcd4 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:32.893056 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fcb76-6364-4375-9e2f-61ab9cdc7586 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:32.933738 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8ed42-84d1-42cd-a4a6-1a3b62b9b5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:32.934376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8ed42-84d1-42cd-a4a6-1a3b62b9b5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:32.934651 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8ed42-84d1-42cd-a4a6-1a3b62b9b5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:32.934933 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8ed42-84d1-42cd-a4a6-1a3b62b9b5e3 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:32.935752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d8ed42-84d1-42cd-a4a6-1a3b62b9b5e3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:32.936079 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d8ed42-84d1-42cd-a4a6-1a3b62b9b5e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:32.937538 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d8ed42-84d1-42cd-a4a6-1a3b62b9b5e3 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:32.955858 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 331/681] 10.222.0.121 () {56 vars in 1140 bytes} [Thu Aug 4 10:19:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:32.957189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd09475-a592-4723-b1d4-f8af9dad5167 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:32.957480 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd09475-a592-4723-b1d4-f8af9dad5167 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:32.957756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd09475-a592-4723-b1d4-f8af9dad5167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:32.976485 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0cd09475-a592-4723-b1d4-f8af9dad5167 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:32.977747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0cd09475-a592-4723-b1d4-f8af9dad5167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:32.982369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:33.001740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e509ac9-102e-4856-9e10-f39478a6bcd4 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:33.041848 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123f30c-e8f4-4bcc-acdb-9a58370d5438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:33.042398 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123f30c-e8f4-4bcc-acdb-9a58370d5438 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:33.042682 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123f30c-e8f4-4bcc-acdb-9a58370d5438 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:33.042950 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123f30c-e8f4-4bcc-acdb-9a58370d5438 None admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:33.043737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123f30c-e8f4-4bcc-acdb-9a58370d5438 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:33.044078 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123f30c-e8f4-4bcc-acdb-9a58370d5438 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:33.045563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123f30c-e8f4-4bcc-acdb-9a58370d5438 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:33.052250 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 332/682] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:32 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:19:33.053639 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd09475-a592-4723-b1d4-f8af9dad5167 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:33.073001 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 351/683] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:19:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:33.073001 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123f30c-e8f4-4bcc-acdb-9a58370d5438 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:33.082169 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:33.169625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd09475-a592-4723-b1d4-f8af9dad5167 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:33.181468 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123f30c-e8f4-4bcc-acdb-9a58370d5438 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:33.209081 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de2b9d-8379-4018-9c89-b92a0e843723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:33.209724 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de2b9d-8379-4018-9c89-b92a0e843723 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:33.210007 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de2b9d-8379-4018-9c89-b92a0e843723 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:33.210280 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de2b9d-8379-4018-9c89-b92a0e843723 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:33.211050 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de2b9d-8379-4018-9c89-b92a0e843723 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:33.211356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de2b9d-8379-4018-9c89-b92a0e843723 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:33.213675 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de2b9d-8379-4018-9c89-b92a0e843723 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:33.223244 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3dcd6-b37c-480d-86d4-024867cfda6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:33.224180 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3dcd6-b37c-480d-86d4-024867cfda6f None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:33.224702 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3dcd6-b37c-480d-86d4-024867cfda6f None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:33.225160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3dcd6-b37c-480d-86d4-024867cfda6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:33.225665 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 333/684] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:19:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:33.226484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de2b9d-8379-4018-9c89-b92a0e843723 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:33.247733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:33.305694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3dcd6-b37c-480d-86d4-024867cfda6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:33.307127 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3dcd6-b37c-480d-86d4-024867cfda6f None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:33.314414 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:33.338078 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de2b9d-8379-4018-9c89-b92a0e843723 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:33.378959 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb1205-490c-4050-a5cd-cdddb07be8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:33.379783 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb1205-490c-4050-a5cd-cdddb07be8dc None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:33.380103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb1205-490c-4050-a5cd-cdddb07be8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:33.380375 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb1205-490c-4050-a5cd-cdddb07be8dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:33.381295 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb1205-490c-4050-a5cd-cdddb07be8dc None admin] RBAC: Adding query filter params (role.id=2edb704d42834db6acd295fa3a8be5d5, scope.project.id=22768aba760b4307aab131d1312b2bf6, user.id=c19bb76d0e2544299916007fbfa62578) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:33.381527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb1205-490c-4050-a5cd-cdddb07be8dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:33.382974 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb1205-490c-4050-a5cd-cdddb07be8dc None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:33.392324 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 334/685] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:19:33 2022] GET /identity/v3/role_assignments?user.id=c19bb76d0e2544299916007fbfa62578&scope.project.id=22768aba760b4307aab131d1312b2bf6&role.id=2edb704d42834db6acd295fa3a8be5d5 => generated 610 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:33.397874 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb1205-490c-4050-a5cd-cdddb07be8dc None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:33.423328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 352/686] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:19:33 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:19:33.424828 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3dcd6-b37c-480d-86d4-024867cfda6f None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:33.488705 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:33.560683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3dcd6-b37c-480d-86d4-024867cfda6f None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:33.565351 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb1205-490c-4050-a5cd-cdddb07be8dc None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:33.600878 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd82baa-6641-4b9d-bd61-1270d49751ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:33.601585 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd82baa-6641-4b9d-bd61-1270d49751ed None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:33.601879 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd82baa-6641-4b9d-bd61-1270d49751ed None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:33.602163 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd82baa-6641-4b9d-bd61-1270d49751ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:33.603130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd82baa-6641-4b9d-bd61-1270d49751ed None admin] RBAC: Adding query filter params (role.id=2edb704d42834db6acd295fa3a8be5d5, scope.project.id=22768aba760b4307aab131d1312b2bf6, user.id=90c7eaf4500f4ef496ba72405522006d) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:33.603451 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd82baa-6641-4b9d-bd61-1270d49751ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:33.604996 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd82baa-6641-4b9d-bd61-1270d49751ed None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:33.613380 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 353/687] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:19:33 2022] GET /identity/v3/role_assignments?user.id=90c7eaf4500f4ef496ba72405522006d&scope.project.id=22768aba760b4307aab131d1312b2bf6&role.id=2edb704d42834db6acd295fa3a8be5d5 => generated 610 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:33.614332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd82baa-6641-4b9d-bd61-1270d49751ed None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:33.630507 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d54a8-6c8b-43ea-ae02-e228e1ef63c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:33.631267 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d54a8-6c8b-43ea-ae02-e228e1ef63c1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:33.631560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d54a8-6c8b-43ea-ae02-e228e1ef63c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:33.631842 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d54a8-6c8b-43ea-ae02-e228e1ef63c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:33.657550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d54a8-6c8b-43ea-ae02-e228e1ef63c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:33.659085 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d54a8-6c8b-43ea-ae02-e228e1ef63c1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:33.661435 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:33.679585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-330d54a8-6c8b-43ea-ae02-e228e1ef63c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:33.679585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:33.679585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:33.679585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:33.679585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:33.679585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:33.679585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:33.679585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:33.679585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:33.679585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:33.679585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:33.679585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:33.679585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:33.679585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:33.679585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:33.680903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:33.680903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:33.680903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:33.680903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:33.680903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:33.680903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:33.680903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:33.680903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:33.680903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:33.680903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:33.680903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:33.680903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:33.680903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:33.680903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:33.680903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:33.680903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:33.680903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:33.681830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:33.681830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:33.681830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:33.681830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:33.683038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 335/688] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:33.693321 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d54a8-6c8b-43ea-ae02-e228e1ef63c1 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:33.775182 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd82baa-6641-4b9d-bd61-1270d49751ed None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:33.786936 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:33.825543 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3eb144-0dc1-491d-9c1a-f86f291c6e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:33.826365 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3eb144-0dc1-491d-9c1a-f86f291c6e38 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:33.826634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3eb144-0dc1-491d-9c1a-f86f291c6e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:33.827005 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3eb144-0dc1-491d-9c1a-f86f291c6e38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:33.840596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3eb144-0dc1-491d-9c1a-f86f291c6e38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:33.842140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3eb144-0dc1-491d-9c1a-f86f291c6e38 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:33.863291 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d54a8-6c8b-43ea-ae02-e228e1ef63c1 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:33.867611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-de3eb144-0dc1-491d-9c1a-f86f291c6e38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:33.867611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:33.867611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:33.867611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:33.867611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:33.867611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:33.867611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:33.867611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:33.867611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:33.867611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:33.867611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:33.867611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:33.867611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:33.867611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:33.867611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:33.868659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:33.868659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:33.868659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:33.868659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:33.868659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:33.868659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:33.868659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:33.868659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:33.868659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:33.868659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:33.868659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:33.868659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:33.868659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:33.868659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:33.868659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:33.868659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:33.868659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:33.869599 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:33.869599 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:33.869599 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:33.869599 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:33.876168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 354/689] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:33.880555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3eb144-0dc1-491d-9c1a-f86f291c6e38 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:33.909247 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f3f72-f118-48a4-818f-d362ef4c0191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:33.910747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f3f72-f118-48a4-818f-d362ef4c0191 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:33.911293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f3f72-f118-48a4-818f-d362ef4c0191 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:33.911908 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f3f72-f118-48a4-818f-d362ef4c0191 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:33.913108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f3f72-f118-48a4-818f-d362ef4c0191 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:33.913631 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f3f72-f118-48a4-818f-d362ef4c0191 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:33.915357 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f3f72-f118-48a4-818f-d362ef4c0191 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:33.931213 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 336/690] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:33 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:33.937238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f3f72-f118-48a4-818f-d362ef4c0191 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:34.000974 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:34.002345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3eb144-0dc1-491d-9c1a-f86f291c6e38 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:34.044949 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edaca7-856e-483a-8bd0-a9257c956c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:34.045612 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edaca7-856e-483a-8bd0-a9257c956c8a None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:34.045900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edaca7-856e-483a-8bd0-a9257c956c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:34.046180 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edaca7-856e-483a-8bd0-a9257c956c8a None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:34.046972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7edaca7-856e-483a-8bd0-a9257c956c8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:34.047286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7edaca7-856e-483a-8bd0-a9257c956c8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:34.048951 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7edaca7-856e-483a-8bd0-a9257c956c8a None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:34.055888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f3f72-f118-48a4-818f-d362ef4c0191 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:34.063680 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 355/691] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:33 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:34.074240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edaca7-856e-483a-8bd0-a9257c956c8a None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:34.101288 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:34.102421 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1012770-81f0-4f95-99da-1443be9d23b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:34.103247 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1012770-81f0-4f95-99da-1443be9d23b5 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:34.103712 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1012770-81f0-4f95-99da-1443be9d23b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:34.104195 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1012770-81f0-4f95-99da-1443be9d23b5 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:34.105316 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1012770-81f0-4f95-99da-1443be9d23b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:34.107062 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1012770-81f0-4f95-99da-1443be9d23b5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:34.134211 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 337/692] 10.222.0.121 () {60 vars in 1137 bytes} [Thu Aug 4 10:19:33 2022] POST /identity/v3/users => generated 255 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:19:34.200304 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edaca7-856e-483a-8bd0-a9257c956c8a None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:34.241699 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795f92c-a57b-482d-ab38-aa59b7b90fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:34.242429 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795f92c-a57b-482d-ab38-aa59b7b90fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:34.242725 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795f92c-a57b-482d-ab38-aa59b7b90fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:34.243002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795f92c-a57b-482d-ab38-aa59b7b90fd1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:34.251693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795f92c-a57b-482d-ab38-aa59b7b90fd1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:34.253300 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795f92c-a57b-482d-ab38-aa59b7b90fd1 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:34.266898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-1795f92c-a57b-482d-ab38-aa59b7b90fd1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:34.266898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:34.266898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:34.266898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:34.266898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:34.266898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:34.266898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:34.266898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:34.266898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:34.266898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:34.266898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:34.266898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:34.266898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:19:34.266898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:19:34.266898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:19:34.268036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:19:34.268036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:34.268036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:34.268036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:34.268036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:34.268036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:34.268036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:34.268036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:34.268036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:34.268036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:34.268036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:34.268036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:34.268036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:34.268036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:34.268036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:34.268036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:34.268036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:34.268989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:19:34.268989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:19:34.268989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:19:34.268989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:19:34.268989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:19:34.268989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:19:34.268989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:34.268989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:34.272117 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 356/693] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:34.274927 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1012770-81f0-4f95-99da-1443be9d23b5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:34.433776 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1012770-81f0-4f95-99da-1443be9d23b5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:34.446733 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:34.474025 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3d4be-09b2-44e6-95a3-19835c2fb11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:34.474783 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3d4be-09b2-44e6-95a3-19835c2fb11f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:34.475066 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3d4be-09b2-44e6-95a3-19835c2fb11f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:34.475337 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3d4be-09b2-44e6-95a3-19835c2fb11f None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:34.476139 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3d4be-09b2-44e6-95a3-19835c2fb11f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:34.476522 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3d4be-09b2-44e6-95a3-19835c2fb11f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:34.477993 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3d4be-09b2-44e6-95a3-19835c2fb11f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:34.492987 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 338/694] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:19:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:34.499488 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795f92c-a57b-482d-ab38-aa59b7b90fd1 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:34.543118 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c586b2-9972-4b11-8434-6ea9935ad48d None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:34.543456 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c586b2-9972-4b11-8434-6ea9935ad48d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:34.543684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c586b2-9972-4b11-8434-6ea9935ad48d None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:34.545424 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 339/695] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:34.551376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd4091-e67a-425c-bf25-42b4bab5fb89 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:34.552213 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd4091-e67a-425c-bf25-42b4bab5fb89 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:34.552356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd4091-e67a-425c-bf25-42b4bab5fb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:34.584762 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-bdfd4091-e67a-425c-bf25-42b4bab5fb89 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:34.586046 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bdfd4091-e67a-425c-bf25-42b4bab5fb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:34.606637 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:34.637746 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795f92c-a57b-482d-ab38-aa59b7b90fd1 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:34.684316 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b307a6e-f512-448e-9324-b609cbe11ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:34.685681 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b307a6e-f512-448e-9324-b609cbe11ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:34.686445 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b307a6e-f512-448e-9324-b609cbe11ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:34.686996 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b307a6e-f512-448e-9324-b609cbe11ce8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:34.688584 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b307a6e-f512-448e-9324-b609cbe11ce8 None admin] RBAC: Adding query filter params (role.id=2edb704d42834db6acd295fa3a8be5d5, scope.project.id=22768aba760b4307aab131d1312b2bf6, user.id=e16e26784601425c9883acd91f65becc) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:34.689467 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b307a6e-f512-448e-9324-b609cbe11ce8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:34.691484 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b307a6e-f512-448e-9324-b609cbe11ce8 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:34.696831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 340/696] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:34 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:19:34.705517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 357/697] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:19:34 2022] GET /identity/v3/role_assignments?user.id=e16e26784601425c9883acd91f65becc&scope.project.id=22768aba760b4307aab131d1312b2bf6&role.id=2edb704d42834db6acd295fa3a8be5d5 => generated 610 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:34.736115 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd4091-e67a-425c-bf25-42b4bab5fb89 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:34.897950 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:34.902326 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd4091-e67a-425c-bf25-42b4bab5fb89 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:34.946460 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f58892-dc08-4276-b717-d2c082a143ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:34.947305 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f58892-dc08-4276-b717-d2c082a143ba None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:34.947441 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f58892-dc08-4276-b717-d2c082a143ba None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:34.947720 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f58892-dc08-4276-b717-d2c082a143ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:34.961333 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f58892-dc08-4276-b717-d2c082a143ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:34.966283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-c4f58892-dc08-4276-b717-d2c082a143ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:19:34.966283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:34.966283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:34.966283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:34.966283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:34.966283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:34.966283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:34.966283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:34.966283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:34.966283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:34.966283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:34.966283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:34.966283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:34.966283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:34.966283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:34.966283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:34.967311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:34.967311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:34.967311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:34.967311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:34.967311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:34.967311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:34.967311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:34.967311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:34.967311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:34.967311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:34.967311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:34.967311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:34.967311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:34.967311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:34.967311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:34.967311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:34.968437 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:34.968437 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:34.968437 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:34.968437 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:34.968437 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:34.968437 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:34.968437 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:34.968437 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:34.968437 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:34.968437 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:34.968437 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:19:34.968437 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:34.968437 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 341/698] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:19:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:34.973876 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b307a6e-f512-448e-9324-b609cbe11ce8 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:35.118677 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b307a6e-f512-448e-9324-b609cbe11ce8 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:35.165711 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d42d0a-004b-4c26-b121-ce7e3c741467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:35.166675 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d42d0a-004b-4c26-b121-ce7e3c741467 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:35.167184 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d42d0a-004b-4c26-b121-ce7e3c741467 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:35.167618 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d42d0a-004b-4c26-b121-ce7e3c741467 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:35.168553 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d42d0a-004b-4c26-b121-ce7e3c741467 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:35.169018 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d42d0a-004b-4c26-b121-ce7e3c741467 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:35.170613 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d42d0a-004b-4c26-b121-ce7e3c741467 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:35.183238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 358/699] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:19:34 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:35.190026 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f58892-dc08-4276-b717-d2c082a143ba None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:35.305882 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f58892-dc08-4276-b717-d2c082a143ba None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:35.342289 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6f8e4-a61a-4ce5-8dfb-a1da3b9ca26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:35.343262 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6f8e4-a61a-4ce5-8dfb-a1da3b9ca26e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:35.343732 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6f8e4-a61a-4ce5-8dfb-a1da3b9ca26e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:35.344189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6f8e4-a61a-4ce5-8dfb-a1da3b9ca26e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:35.356133 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6f8e4-a61a-4ce5-8dfb-a1da3b9ca26e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:35.357928 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6f8e4-a61a-4ce5-8dfb-a1da3b9ca26e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:35.373881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-25f6f8e4-a61a-4ce5-8dfb-a1da3b9ca26e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:35.373881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:35.373881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:35.373881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:35.373881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:35.373881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:35.373881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:35.373881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:35.373881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:35.373881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:35.373881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:35.373881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:35.373881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:35.373881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:35.373881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:35.375131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:35.375131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:35.375131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:35.375131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:35.375131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:35.375131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:35.375131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:35.375131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:35.375131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:35.375131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:35.375131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:35.375131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:35.375131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:35.375131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:35.375131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:35.375131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:35.375131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:35.376318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:35.376318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:35.376318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:35.376318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:35.378322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 342/700] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:35.385472 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d42d0a-004b-4c26-b121-ce7e3c741467 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:35.407861 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:35.545069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d42d0a-004b-4c26-b121-ce7e3c741467 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:35.587994 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd0709-fcf7-488c-9584-261714f770d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:35.588670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd0709-fcf7-488c-9584-261714f770d4 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:35.588954 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd0709-fcf7-488c-9584-261714f770d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:35.589228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd0709-fcf7-488c-9584-261714f770d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:35.590021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd0709-fcf7-488c-9584-261714f770d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:35.590334 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd0709-fcf7-488c-9584-261714f770d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:35.591796 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd0709-fcf7-488c-9584-261714f770d4 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:35.614372 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 359/701] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:35.624067 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6f8e4-a61a-4ce5-8dfb-a1da3b9ca26e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:35.652387 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:35.777398 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6f8e4-a61a-4ce5-8dfb-a1da3b9ca26e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:35.822299 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220546a0-9562-4787-a931-0276d253136b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:35.822981 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220546a0-9562-4787-a931-0276d253136b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:35.823270 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220546a0-9562-4787-a931-0276d253136b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:35.823552 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220546a0-9562-4787-a931-0276d253136b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:35.837783 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220546a0-9562-4787-a931-0276d253136b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:35.839200 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220546a0-9562-4787-a931-0276d253136b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:35.860509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-220546a0-9562-4787-a931-0276d253136b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:19:35.860509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:35.860509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:35.860509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:35.860509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:35.860509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:35.860509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:35.860509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:35.860509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:35.860509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:35.860509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:35.860509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:35.860509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:19:35.860509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:19:35.860509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:19:35.860509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:19:35.861638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:35.861638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:35.861638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:19:35.861638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:35.861638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:19:35.861638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:35.861638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:35.861638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:35.861638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:35.861638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:35.861638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:35.861638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:35.861638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:35.861638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:35.861638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:35.861638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:35.861638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:35.861638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:35.863067 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:35.863067 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:35.863067 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:19:35.863067 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:19:35.863067 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:19:35.863067 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:19:35.863067 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:19:35.863067 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:19:35.863067 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:19:35.863067 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:35.863067 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 343/702] 10.222.0.121 () {56 vars in 1098 bytes} [Thu Aug 4 10:19:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:35.869185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd0709-fcf7-488c-9584-261714f770d4 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:35.997729 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:36.060867 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd0709-fcf7-488c-9584-261714f770d4 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:36.113970 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7927f-07cb-49f2-b9e2-d28f97b1d44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:36.114533 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7927f-07cb-49f2-b9e2-d28f97b1d44f None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:36.114818 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7927f-07cb-49f2-b9e2-d28f97b1d44f None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:36.115097 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7927f-07cb-49f2-b9e2-d28f97b1d44f None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:36.116105 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7927f-07cb-49f2-b9e2-d28f97b1d44f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:36.116420 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7927f-07cb-49f2-b9e2-d28f97b1d44f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:36.118007 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7927f-07cb-49f2-b9e2-d28f97b1d44f None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:36.143331 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 360/703] 10.222.0.121 () {56 vars in 1138 bytes} [Thu Aug 4 10:19:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:36.149414 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220546a0-9562-4787-a931-0276d253136b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:36.181831 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da660527-5895-4ac9-b993-eaa24495d91d None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:36.181831 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da660527-5895-4ac9-b993-eaa24495d91d None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:36.181831 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da660527-5895-4ac9-b993-eaa24495d91d None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:36.182930 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 361/704] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:36 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:36.188342 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027801a9-8599-4b22-93e4-f167fa60a3a0 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:36.188692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027801a9-8599-4b22-93e4-f167fa60a3a0 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:36.189072 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027801a9-8599-4b22-93e4-f167fa60a3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:36.212127 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-027801a9-8599-4b22-93e4-f167fa60a3a0 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:36.219992 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-027801a9-8599-4b22-93e4-f167fa60a3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:36.277301 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:36.337453 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220546a0-9562-4787-a931-0276d253136b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:36.348807 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 362/705] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:36 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 10:19:36.388659 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c85fd58-e470-4012-b72e-c8830393cd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:36.389182 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85fd58-e470-4012-b72e-c8830393cd43 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:36.389499 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85fd58-e470-4012-b72e-c8830393cd43 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:36.389759 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85fd58-e470-4012-b72e-c8830393cd43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:36.425306 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027801a9-8599-4b22-93e4-f167fa60a3a0 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:36.513649 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:36.517321 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c85fd58-e470-4012-b72e-c8830393cd43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:36.518920 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c85fd58-e470-4012-b72e-c8830393cd43 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:36.571188 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027801a9-8599-4b22-93e4-f167fa60a3a0 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:36.615541 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:36.619394 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e8bb4-19ce-46c8-998e-52e43f8d5c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:36.620469 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e8bb4-19ce-46c8-998e-52e43f8d5c70 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:36.620901 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e8bb4-19ce-46c8-998e-52e43f8d5c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:36.621354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e8bb4-19ce-46c8-998e-52e43f8d5c70 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:36.622457 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e8bb4-19ce-46c8-998e-52e43f8d5c70 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:36.624402 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e8bb4-19ce-46c8-998e-52e43f8d5c70 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:36.640943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-0a1e8bb4-19ce-46c8-998e-52e43f8d5c70 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 10:19:36.640943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:36.640943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:19:36.640943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:19:36.640943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:19:36.640943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:19:36.640943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:19:36.640943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:19:36.640943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 10:19:36.640943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:36.640943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:19:36.640943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:36.640943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:36.640943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:36.640943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:36.640943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:36.642015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:36.642015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:36.642015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:36.642015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:36.642015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:36.642015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:36.642015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:36.642015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:19:36.642015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:19:36.642015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:19:36.642015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:19:36.642015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:36.642015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:36.642015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:36.642015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:36.642015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:36.642899 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:36.642899 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:36.642899 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:36.642899 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:36.642899 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:36.642899 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:36.642899 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:36.642899 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:36.642899 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:36.642899 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:36.642899 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:36.642899 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:19:36.642899 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:19:36.642899 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 10:19:36.642899 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:36.642899 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 363/706] 10.222.0.121 () {56 vars in 1134 bytes} [Thu Aug 4 10:19:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:19:36.643825 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8c54b-7550-41a4-aaf8-8ec65658e2df None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:36.644479 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8c54b-7550-41a4-aaf8-8ec65658e2df None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:36.644479 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8c54b-7550-41a4-aaf8-8ec65658e2df None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:36.646765 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 364/707] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:36 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:36.648528 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8c54b-7550-41a4-aaf8-8ec65658e2df None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:36.665190 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 344/708] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:19:36 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:19:36.666691 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24054ff-fbca-48bc-83b2-17dbf9694eaa None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:36.666985 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24054ff-fbca-48bc-83b2-17dbf9694eaa None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:36.667396 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24054ff-fbca-48bc-83b2-17dbf9694eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:36.688061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c24054ff-fbca-48bc-83b2-17dbf9694eaa None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:36.689323 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c24054ff-fbca-48bc-83b2-17dbf9694eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:36.776121 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 345/709] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:36 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:19:36.777175 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24054ff-fbca-48bc-83b2-17dbf9694eaa None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:36.806286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8c54b-7550-41a4-aaf8-8ec65658e2df None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:36.826736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:36.846925 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af66256-e291-4978-92a3-be52ff60b3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:36.847575 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af66256-e291-4978-92a3-be52ff60b3a6 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:36.847861 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af66256-e291-4978-92a3-be52ff60b3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:36.848167 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af66256-e291-4978-92a3-be52ff60b3a6 None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:36.848865 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af66256-e291-4978-92a3-be52ff60b3a6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:36.849178 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af66256-e291-4978-92a3-be52ff60b3a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:36.850664 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af66256-e291-4978-92a3-be52ff60b3a6 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:36.855957 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 365/710] 10.222.0.121 () {56 vars in 1129 bytes} [Thu Aug 4 10:19:36 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:36.856820 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af66256-e291-4978-92a3-be52ff60b3a6 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:36.911366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24054ff-fbca-48bc-83b2-17dbf9694eaa None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:36.913103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:36.954536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a6c53-8829-43e7-8f64-c89681911b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:36.955676 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a6c53-8829-43e7-8f64-c89681911b4b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:36.956144 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a6c53-8829-43e7-8f64-c89681911b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:36.956705 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a6c53-8829-43e7-8f64-c89681911b4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:36.965729 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af66256-e291-4978-92a3-be52ff60b3a6 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:36.969599 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702a6c53-8829-43e7-8f64-c89681911b4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:36.971409 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702a6c53-8829-43e7-8f64-c89681911b4b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:36.989263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-702a6c53-8829-43e7-8f64-c89681911b4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:36.989263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:36.989263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:36.989263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:36.989263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:36.989263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:36.989263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:36.989263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:36.989263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:36.989263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:36.989263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:36.989263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:36.989263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:36.989263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:36.989263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:36.990583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:36.990583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:36.990583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:36.990583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:36.990583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:36.990583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:36.990583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:36.990583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:36.990583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:36.990583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:36.990583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:36.990583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:36.990583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:36.990583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:36.990583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:36.990583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:36.990583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:36.991738 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:36.991738 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:36.991738 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:36.991738 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:36.993948 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 346/711] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:36.995306 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a6c53-8829-43e7-8f64-c89681911b4b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:37.003381 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd626de-6fe7-42f8-8ac3-d9b03640faaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:37.004212 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd626de-6fe7-42f8-8ac3-d9b03640faaf None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:37.004392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd626de-6fe7-42f8-8ac3-d9b03640faaf None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:37.004712 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd626de-6fe7-42f8-8ac3-d9b03640faaf None admin] PATH_INFO: `/v3/services/network` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:37.005364 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd626de-6fe7-42f8-8ac3-d9b03640faaf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:37.006826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd626de-6fe7-42f8-8ac3-d9b03640faaf None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:37.017084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-2bd626de-6fe7-42f8-8ac3-d9b03640faaf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 10:19:37.017084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:37.017084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:19:37.017084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:19:37.017084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:19:37.017084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:19:37.017084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:19:37.017084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:19:37.017084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 10:19:37.017084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:37.017084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:19:37.017084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:37.017084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:37.017084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:37.017084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:37.017084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:37.017084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:37.018295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:37.018295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:37.018295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:37.018295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:37.018295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:37.018295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:37.018295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:19:37.018295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:19:37.018295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:19:37.018295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:19:37.018295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:37.018295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:37.018295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:37.018295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:37.018295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:37.018295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:37.019905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:37.019905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:37.019905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:37.019905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:37.019905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:37.019905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:37.019905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:37.019905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:37.019905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:37.019905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:37.019905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:19:37.019905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:19:37.019905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 10:19:37.019905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:37.023316 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 366/712] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:36 2022] GET /identity/v3/services/network => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:37.024441 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd626de-6fe7-42f8-8ac3-d9b03640faaf None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:37.076961 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:37.127868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a6c53-8829-43e7-8f64-c89681911b4b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:37.146903 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd626de-6fe7-42f8-8ac3-d9b03640faaf None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:37.169756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1501a-ed79-4900-ab65-a6d99bb886ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:37.171267 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1501a-ed79-4900-ab65-a6d99bb886ef None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:37.171854 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1501a-ed79-4900-ab65-a6d99bb886ef None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:37.172399 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1501a-ed79-4900-ab65-a6d99bb886ef None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:37.173371 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1501a-ed79-4900-ab65-a6d99bb886ef None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:37.173908 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1501a-ed79-4900-ab65-a6d99bb886ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:37.175700 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1501a-ed79-4900-ab65-a6d99bb886ef None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:37.183119 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 347/713] 10.222.0.121 () {56 vars in 1129 bytes} [Thu Aug 4 10:19:36 2022] GET /identity/v3/services?type=block-storage => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:37.184677 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1501a-ed79-4900-ab65-a6d99bb886ef None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:37.191902 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953c8b8-b6c3-4abe-9bf0-17be22e7d4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:37.192621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953c8b8-b6c3-4abe-9bf0-17be22e7d4ec None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:37.192909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953c8b8-b6c3-4abe-9bf0-17be22e7d4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:37.193190 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953c8b8-b6c3-4abe-9bf0-17be22e7d4ec None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:37.194025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d953c8b8-b6c3-4abe-9bf0-17be22e7d4ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:37.194342 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d953c8b8-b6c3-4abe-9bf0-17be22e7d4ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:37.195888 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d953c8b8-b6c3-4abe-9bf0-17be22e7d4ec None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:37.213941 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 367/714] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:37 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:37.216087 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b3e09-6a94-4417-bb59-e2fc90af17da None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:37.216416 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b3e09-6a94-4417-bb59-e2fc90af17da None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:37.216772 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b3e09-6a94-4417-bb59-e2fc90af17da None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:37.218375 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:37.220647 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 368/715] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:37 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:37.221860 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b3e09-6a94-4417-bb59-e2fc90af17da None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:37.369660 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1501a-ed79-4900-ab65-a6d99bb886ef None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:37.381351 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b3e09-6a94-4417-bb59-e2fc90af17da None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:37.392085 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:37.431090 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729c646-a2e2-486e-a373-d7531ca03af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:37.431615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729c646-a2e2-486e-a373-d7531ca03af6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:37.431900 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729c646-a2e2-486e-a373-d7531ca03af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:37.432287 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729c646-a2e2-486e-a373-d7531ca03af6 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:37.433008 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0729c646-a2e2-486e-a373-d7531ca03af6 None admin] RBAC: Adding query filter params (name=network) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:37.433319 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0729c646-a2e2-486e-a373-d7531ca03af6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:37.434767 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0729c646-a2e2-486e-a373-d7531ca03af6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:37.443983 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 348/716] 10.222.0.121 () {56 vars in 1117 bytes} [Thu Aug 4 10:19:37 2022] GET /identity/v3/services?name=network => generated 125 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:37.443983 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c87e32-9f4b-4648-9920-c0caf49f02ac None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:37.443983 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c87e32-9f4b-4648-9920-c0caf49f02ac None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:37.443983 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c87e32-9f4b-4648-9920-c0caf49f02ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:37.450060 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c5e64-c1fe-4f61-9b1d-8e3068b42f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:37.452783 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c5e64-c1fe-4f61-9b1d-8e3068b42f46 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:37.453550 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c5e64-c1fe-4f61-9b1d-8e3068b42f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:37.454095 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c5e64-c1fe-4f61-9b1d-8e3068b42f46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:37.463167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e7c87e32-9f4b-4648-9920-c0caf49f02ac None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:37.464461 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e7c87e32-9f4b-4648-9920-c0caf49f02ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:37.465582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c5e64-c1fe-4f61-9b1d-8e3068b42f46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:37.467472 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c5e64-c1fe-4f61-9b1d-8e3068b42f46 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:37.484200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-727c5e64-c1fe-4f61-9b1d-8e3068b42f46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:37.484200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:37.484200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:37.484200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:37.484200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:37.484200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:37.484200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:37.484200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:37.484200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:37.484200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:37.484200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:37.484200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:37.484200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:19:37.484200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:19:37.484200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:19:37.486268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:19:37.486268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:37.486268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:37.486268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:37.486268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:37.486268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:37.486268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:37.486268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:37.486268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:37.486268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:37.486268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:37.486268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:37.486268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:37.486268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:37.486268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:37.486268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:37.486268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:37.488453 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:19:37.488453 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:19:37.488453 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:19:37.488453 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:19:37.488453 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:19:37.488453 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:19:37.488453 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:37.488453 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:37.491668 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 369/717] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:37.493452 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c5e64-c1fe-4f61-9b1d-8e3068b42f46 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:37.541744 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 349/718] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:37 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:19:37.542849 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c87e32-9f4b-4648-9920-c0caf49f02ac None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:37.560809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:37.633773 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c5e64-c1fe-4f61-9b1d-8e3068b42f46 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:37.674361 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c87e32-9f4b-4648-9920-c0caf49f02ac None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:37.686388 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b0233-265d-460a-98e4-3216e7e74ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:37.687300 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b0233-265d-460a-98e4-3216e7e74ddc None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:37.687774 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b0233-265d-460a-98e4-3216e7e74ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:37.688287 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b0233-265d-460a-98e4-3216e7e74ddc None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:37.689485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b0233-265d-460a-98e4-3216e7e74ddc None admin] RBAC: Adding query filter params (type=network) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:37.690018 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b0233-265d-460a-98e4-3216e7e74ddc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:37.692105 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b0233-265d-460a-98e4-3216e7e74ddc None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:37.701579 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 370/719] 10.222.0.121 () {56 vars in 1117 bytes} [Thu Aug 4 10:19:37 2022] GET /identity/v3/services?type=network => generated 125 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:37.703473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6568e9a-c451-4b58-8a34-e8dcdea576fc None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:37.703930 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6568e9a-c451-4b58-8a34-e8dcdea576fc None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:37.704406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6568e9a-c451-4b58-8a34-e8dcdea576fc None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:37.706259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 371/720] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:37 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:37.706970 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6568e9a-c451-4b58-8a34-e8dcdea576fc None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:37.743650 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688f2a9-5b67-4fbc-8dd5-cdbc900e1539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:37.744476 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688f2a9-5b67-4fbc-8dd5-cdbc900e1539 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:37.744605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688f2a9-5b67-4fbc-8dd5-cdbc900e1539 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:37.744917 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688f2a9-5b67-4fbc-8dd5-cdbc900e1539 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:37.745720 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688f2a9-5b67-4fbc-8dd5-cdbc900e1539 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:37.745993 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688f2a9-5b67-4fbc-8dd5-cdbc900e1539 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:37.747591 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688f2a9-5b67-4fbc-8dd5-cdbc900e1539 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:37.757586 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:37.779024 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 350/721] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:19:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:37.779024 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546be3fa-76bf-430d-8590-02cb4617e06e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:37.779024 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546be3fa-76bf-430d-8590-02cb4617e06e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:37.779024 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546be3fa-76bf-430d-8590-02cb4617e06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:37.808654 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-546be3fa-76bf-430d-8590-02cb4617e06e None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:37.810622 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-546be3fa-76bf-430d-8590-02cb4617e06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:37.871043 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6568e9a-c451-4b58-8a34-e8dcdea576fc None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:37.915021 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 351/722] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:37 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:19:37.915815 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546be3fa-76bf-430d-8590-02cb4617e06e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:37.918418 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:37.920867 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ab0b0-5822-445b-be46-7ba6683f0b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:37.921576 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ab0b0-5822-445b-be46-7ba6683f0b84 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:37.921958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ab0b0-5822-445b-be46-7ba6683f0b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:37.922139 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ab0b0-5822-445b-be46-7ba6683f0b84 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:37.953407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09ab0b0-5822-445b-be46-7ba6683f0b84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:37.959011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-d09ab0b0-5822-445b-be46-7ba6683f0b84 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:37.959011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:37.959011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:37.959011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:37.959011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:37.959011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:37.959011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:37.959011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:37.959011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:37.959011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:37.959011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:37.959011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:37.959011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:37.959011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:37.959011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:37.959011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:37.961268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:37.961268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:37.961268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:37.961268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:37.961268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:37.961268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:37.961268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:37.961268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:37.961268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:37.961268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:37.961268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:37.961268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:37.961268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:37.961268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:37.961268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:37.961268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:37.963027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:37.963027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:37.963027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:37.963027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:37.963027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:37.963027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:37.963027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:37.963027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:37.963027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:37.963027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:37.963027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:37.963027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:37.963027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 372/723] 10.222.0.121 () {56 vars in 1107 bytes} [Thu Aug 4 10:19:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:37.963027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ab0b0-5822-445b-be46-7ba6683f0b84 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:38.070123 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546be3fa-76bf-430d-8590-02cb4617e06e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:38.078271 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:38.091462 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ab0b0-5822-445b-be46-7ba6683f0b84 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:38.123022 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaea704-9f04-477c-b0d5-879c37eab5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:38.124680 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaea704-9f04-477c-b0d5-879c37eab5ee None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:38.125158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaea704-9f04-477c-b0d5-879c37eab5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:38.125610 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaea704-9f04-477c-b0d5-879c37eab5ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:38.126597 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaea704-9f04-477c-b0d5-879c37eab5ee None admin] RBAC: Adding query filter params (role.id=7342106a255b4bce8f150726602a8b95, scope.project.id=22768aba760b4307aab131d1312b2bf6, user.id=0d947aeb749546309a8d4b7938087ba0) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:38.127074 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaea704-9f04-477c-b0d5-879c37eab5ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:38.128501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982486f-fc7c-4844-ad94-711231490918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:38.128981 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaea704-9f04-477c-b0d5-879c37eab5ee None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:38.129297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982486f-fc7c-4844-ad94-711231490918 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:38.129583 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982486f-fc7c-4844-ad94-711231490918 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:38.129858 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982486f-fc7c-4844-ad94-711231490918 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:38.137607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 352/724] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:19:37 2022] GET /identity/v3/role_assignments?user.id=0d947aeb749546309a8d4b7938087ba0&scope.project.id=22768aba760b4307aab131d1312b2bf6&role.id=7342106a255b4bce8f150726602a8b95 => generated 260 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:38.139486 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaea704-9f04-477c-b0d5-879c37eab5ee None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:38.142033 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982486f-fc7c-4844-ad94-711231490918 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:38.148175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-7982486f-fc7c-4844-ad94-711231490918 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:19:38.148175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:38.148175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:38.148175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:38.148175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:38.148175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:38.148175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:38.148175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:38.148175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:38.148175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:38.148175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:38.148175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:38.148175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:38.148175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:38.148175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:38.148175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:38.149303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:38.149303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:38.149303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:38.149303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:38.149303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:38.149303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:38.149303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:38.149303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:38.149303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:38.149303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:38.149303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:38.149303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:38.149303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:38.149303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:38.149303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:38.149303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:38.150291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:38.150291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:38.150291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:38.150291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:38.150291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:38.150291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:38.150291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:38.150291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:38.150291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:38.150291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:38.150291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:19:38.150291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:38.150291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 373/725] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:19:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:38.156204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982486f-fc7c-4844-ad94-711231490918 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:38.243999 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:38.274754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982486f-fc7c-4844-ad94-711231490918 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:38.300616 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaea704-9f04-477c-b0d5-879c37eab5ee None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:38.326004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c4e24-6c1b-4245-be61-f8b9cf236ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:38.327717 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c4e24-6c1b-4245-be61-f8b9cf236ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:38.328079 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c4e24-6c1b-4245-be61-f8b9cf236ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:38.328358 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c4e24-6c1b-4245-be61-f8b9cf236ba5 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:38.329133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c4e24-6c1b-4245-be61-f8b9cf236ba5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:38.329443 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c4e24-6c1b-4245-be61-f8b9cf236ba5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:38.330918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c4e24-6c1b-4245-be61-f8b9cf236ba5 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:38.350713 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 374/726] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:19:38 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:19:38.360043 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c4e24-6c1b-4245-be61-f8b9cf236ba5 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:38.367744 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78192eaa-7633-44d5-89da-cda5c23e11ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:38.368430 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78192eaa-7633-44d5-89da-cda5c23e11ea None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:38.368712 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78192eaa-7633-44d5-89da-cda5c23e11ea None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:38.368985 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78192eaa-7633-44d5-89da-cda5c23e11ea None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:38.369733 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78192eaa-7633-44d5-89da-cda5c23e11ea None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:38.370040 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78192eaa-7633-44d5-89da-cda5c23e11ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:38.371458 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78192eaa-7633-44d5-89da-cda5c23e11ea None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:38.388639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:38.396145 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 353/727] 10.222.0.121 () {56 vars in 1108 bytes} [Thu Aug 4 10:19:38 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:38.405155 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78192eaa-7633-44d5-89da-cda5c23e11ea None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:38.555225 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c4e24-6c1b-4245-be61-f8b9cf236ba5 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:38.574050 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78192eaa-7633-44d5-89da-cda5c23e11ea None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:38.603961 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:38.621476 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073473d-ab9a-4e89-b6c8-71975cfd67d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:38.622168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073473d-ab9a-4e89-b6c8-71975cfd67d9 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:38.622457 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073473d-ab9a-4e89-b6c8-71975cfd67d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:38.622738 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073473d-ab9a-4e89-b6c8-71975cfd67d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:38.636612 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e8cea-76b1-405f-8977-54876eb9aeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:38.637316 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e8cea-76b1-405f-8977-54876eb9aeee None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:38.637606 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e8cea-76b1-405f-8977-54876eb9aeee None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:38.637890 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e8cea-76b1-405f-8977-54876eb9aeee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:38.642879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2073473d-ab9a-4e89-b6c8-71975cfd67d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:38.644575 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2073473d-ab9a-4e89-b6c8-71975cfd67d9 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:38.660798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-2073473d-ab9a-4e89-b6c8-71975cfd67d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:38.660798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:38.660798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:38.660798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:38.660798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:38.660798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:38.660798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:38.660798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:38.660798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:38.660798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:38.660798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:38.660798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:38.660798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:38.660798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:38.660798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:38.662038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:38.662038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:38.662038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:38.662038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:38.662038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:38.662038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:38.662038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:38.662038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:38.662038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:38.662038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:38.662038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:38.662038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:38.662038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:38.662038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:38.662038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:38.662038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:38.662038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:38.663194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:38.663194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:38.663194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:38.663194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:38.666100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998e8cea-76b1-405f-8977-54876eb9aeee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:38.666920 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 354/728] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:38.667879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073473d-ab9a-4e89-b6c8-71975cfd67d9 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:38.668449 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998e8cea-76b1-405f-8977-54876eb9aeee None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:38.685711 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-998e8cea-76b1-405f-8977-54876eb9aeee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:38.685711 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:38.685711 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:38.685711 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:38.685711 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:38.685711 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:38.685711 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:38.685711 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:38.685711 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:38.685711 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:38.685711 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:38.685711 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:38.685711 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:38.685711 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:38.685711 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:38.687075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:38.687075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:38.687075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:38.687075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:38.687075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:38.687075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:38.687075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:38.687075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:38.687075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:38.687075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:38.687075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:38.687075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:38.687075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:38.687075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:38.687075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:38.687075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:38.687075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:38.688391 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:38.688391 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:38.688391 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:38.688391 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:38.690460 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 375/729] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:38.697079 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e8cea-76b1-405f-8977-54876eb9aeee None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:38.803744 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073473d-ab9a-4e89-b6c8-71975cfd67d9 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:38.824659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:38.845573 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014df5c6-8fb4-4aa6-8a59-66165f1ab713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:38.846701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014df5c6-8fb4-4aa6-8a59-66165f1ab713 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:38.847303 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014df5c6-8fb4-4aa6-8a59-66165f1ab713 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:38.847797 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014df5c6-8fb4-4aa6-8a59-66165f1ab713 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:38.848854 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014df5c6-8fb4-4aa6-8a59-66165f1ab713 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:38.849416 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014df5c6-8fb4-4aa6-8a59-66165f1ab713 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:38.851272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014df5c6-8fb4-4aa6-8a59-66165f1ab713 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:38.866484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 355/730] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:38 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:38.870682 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e8cea-76b1-405f-8977-54876eb9aeee None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:38.872954 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014df5c6-8fb4-4aa6-8a59-66165f1ab713 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:38.936350 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:38.936596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3bc1ad-8b61-4b6c-9a50-7661c431d488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:38.937136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3bc1ad-8b61-4b6c-9a50-7661c431d488 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:38.937424 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3bc1ad-8b61-4b6c-9a50-7661c431d488 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:38.937725 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3bc1ad-8b61-4b6c-9a50-7661c431d488 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:38.939331 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3bc1ad-8b61-4b6c-9a50-7661c431d488 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:38.939331 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3bc1ad-8b61-4b6c-9a50-7661c431d488 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:38.940293 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3bc1ad-8b61-4b6c-9a50-7661c431d488 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:38.980386 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 376/731] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:38 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:38.985935 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3bc1ad-8b61-4b6c-9a50-7661c431d488 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:39.010512 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014df5c6-8fb4-4aa6-8a59-66165f1ab713 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:39.065995 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58348de1-0d20-4c9d-b243-2e37063fb93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:39.066740 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58348de1-0d20-4c9d-b243-2e37063fb93f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:39.067035 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58348de1-0d20-4c9d-b243-2e37063fb93f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:39.067315 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58348de1-0d20-4c9d-b243-2e37063fb93f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:39.089734 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58348de1-0d20-4c9d-b243-2e37063fb93f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:39.091287 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58348de1-0d20-4c9d-b243-2e37063fb93f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:39.115736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-58348de1-0d20-4c9d-b243-2e37063fb93f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 10:19:39.115736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:39.115736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:39.115736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:39.115736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:39.115736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:39.115736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:39.115736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:39.115736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:39.115736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:39.115736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:39.115736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:39.115736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:19:39.115736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:19:39.115736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:19:39.115736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:19:39.116883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:39.116883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:39.116883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:19:39.116883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:39.116883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:19:39.116883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:39.116883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:39.116883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:39.116883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:39.116883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:39.116883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:39.116883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:39.116883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:39.116883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:39.116883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:39.116883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:39.116883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:39.116883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:39.118644 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:39.118644 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:39.118644 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:19:39.118644 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:19:39.118644 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:19:39.118644 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:19:39.118644 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:19:39.118644 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:19:39.118644 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 10:19:39.118644 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:39.118644 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 356/732] 10.222.0.121 () {56 vars in 1122 bytes} [Thu Aug 4 10:19:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:19:39.125246 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58348de1-0d20-4c9d-b243-2e37063fb93f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:39.262246 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3bc1ad-8b61-4b6c-9a50-7661c431d488 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:39.296636 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:39.312217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45984c9f-9030-41f3-b66f-7ce9ae24a65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:39.313401 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45984c9f-9030-41f3-b66f-7ce9ae24a65d None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:39.314269 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45984c9f-9030-41f3-b66f-7ce9ae24a65d None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:39.314778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45984c9f-9030-41f3-b66f-7ce9ae24a65d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:39.325203 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45984c9f-9030-41f3-b66f-7ce9ae24a65d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:39.327215 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45984c9f-9030-41f3-b66f-7ce9ae24a65d None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:39.342841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-45984c9f-9030-41f3-b66f-7ce9ae24a65d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 10:19:39.342841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:39.342841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:39.342841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:39.342841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:39.342841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:39.342841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:39.342841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:39.342841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:39.342841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:39.342841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:39.342841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:39.342841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:19:39.342841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:19:39.342841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:19:39.342841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:19:39.344029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:39.344029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:39.344029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:19:39.344029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:39.344029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:19:39.344029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:39.344029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:39.344029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:39.344029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:39.344029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:39.344029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:39.344029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:39.344029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:39.344029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:39.344029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:39.344029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:39.344029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:39.344029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:39.345042 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:39.345042 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:39.345042 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:19:39.345042 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:19:39.345042 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:19:39.345042 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:19:39.345042 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:19:39.345042 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:19:39.345042 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 10:19:39.345042 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:39.348279 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 377/733] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:19:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:19:39.366179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45984c9f-9030-41f3-b66f-7ce9ae24a65d None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:39.456605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:39.461728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58348de1-0d20-4c9d-b243-2e37063fb93f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:39.555468 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ad73e-b93e-4da3-9571-da79bd3a7b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:39.555468 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ad73e-b93e-4da3-9571-da79bd3a7b37 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:39.555468 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ad73e-b93e-4da3-9571-da79bd3a7b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:39.555927 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ad73e-b93e-4da3-9571-da79bd3a7b37 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:39.559984 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ad73e-b93e-4da3-9571-da79bd3a7b37 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:39.559984 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ad73e-b93e-4da3-9571-da79bd3a7b37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:39.561883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ad73e-b93e-4da3-9571-da79bd3a7b37 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:39.582506 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45984c9f-9030-41f3-b66f-7ce9ae24a65d None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:39.628721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 357/734] 10.222.0.121 () {56 vars in 1154 bytes} [Thu Aug 4 10:19:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 504 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:39.633856 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ad73e-b93e-4da3-9571-da79bd3a7b37 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:39.640712 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96792609-301e-411a-a7d4-d4e699a3e1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:39.641393 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96792609-301e-411a-a7d4-d4e699a3e1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:39.641679 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96792609-301e-411a-a7d4-d4e699a3e1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:39.641960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96792609-301e-411a-a7d4-d4e699a3e1c6 None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:39.642804 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96792609-301e-411a-a7d4-d4e699a3e1c6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:39.643112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96792609-301e-411a-a7d4-d4e699a3e1c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:39.644684 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96792609-301e-411a-a7d4-d4e699a3e1c6 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:39.694985 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 378/735] 10.222.0.121 () {56 vars in 1140 bytes} [Thu Aug 4 10:19:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:39.703806 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96792609-301e-411a-a7d4-d4e699a3e1c6 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:39.807999 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ad73e-b93e-4da3-9571-da79bd3a7b37 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:39.817262 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:39.846840 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e72825-6d97-4939-975c-e411b1d28d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:39.847491 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e72825-6d97-4939-975c-e411b1d28d87 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:39.847775 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e72825-6d97-4939-975c-e411b1d28d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:39.848082 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e72825-6d97-4939-975c-e411b1d28d87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:39.904563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:39.948752 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96792609-301e-411a-a7d4-d4e699a3e1c6 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:39.989590 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e72825-6d97-4939-975c-e411b1d28d87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:39.993410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e72825-6d97-4939-975c-e411b1d28d87 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:40.008246 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc954b0e-7587-4e3e-bad6-1c64bcc450d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:40.009254 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc954b0e-7587-4e3e-bad6-1c64bcc450d6 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:40.009736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc954b0e-7587-4e3e-bad6-1c64bcc450d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:40.010266 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc954b0e-7587-4e3e-bad6-1c64bcc450d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:40.142307 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc954b0e-7587-4e3e-bad6-1c64bcc450d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:40.144055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc954b0e-7587-4e3e-bad6-1c64bcc450d6 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:40.164389 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 358/736] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:19:39 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 10:19:40.165858 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429bcc7-2f0f-4359-8c72-ef84d3d707ec None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:40.166158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429bcc7-2f0f-4359-8c72-ef84d3d707ec None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:40.166439 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429bcc7-2f0f-4359-8c72-ef84d3d707ec None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:40.168407 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 359/737] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:40.169248 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429bcc7-2f0f-4359-8c72-ef84d3d707ec None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:40.205424 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:40.250786 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:40.282334 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 379/738] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:19:39 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 10:19:40.284629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60996dae-74ea-4216-9b8a-f0c216b4fa08 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:40.285252 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60996dae-74ea-4216-9b8a-f0c216b4fa08 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:40.285746 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60996dae-74ea-4216-9b8a-f0c216b4fa08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:40.308134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-60996dae-74ea-4216-9b8a-f0c216b4fa08 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:40.309801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-60996dae-74ea-4216-9b8a-f0c216b4fa08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:40.316518 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429bcc7-2f0f-4359-8c72-ef84d3d707ec None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:40.363557 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf91636-a02f-4f95-9cc5-730605c1337e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:40.364348 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf91636-a02f-4f95-9cc5-730605c1337e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:40.364641 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf91636-a02f-4f95-9cc5-730605c1337e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:40.364920 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf91636-a02f-4f95-9cc5-730605c1337e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:40.380108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf91636-a02f-4f95-9cc5-730605c1337e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:40.381706 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf91636-a02f-4f95-9cc5-730605c1337e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:40.402591 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-bbf91636-a02f-4f95-9cc5-730605c1337e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:40.402591 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:40.402591 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:40.402591 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:40.402591 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:40.402591 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:40.402591 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:40.402591 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:40.402591 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:40.402591 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:40.402591 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:40.402591 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:40.402591 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:40.402591 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:40.402591 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:40.403721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:40.403721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:40.403721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:40.403721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:40.403721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:40.403721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:40.403721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:40.403721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:40.403721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:40.403721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:40.403721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:40.403721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:40.403721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:40.403721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:40.403721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:40.403721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:40.403721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:40.404802 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:40.404802 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:40.404802 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:40.404802 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:40.408140 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 360/739] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:40.409486 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf91636-a02f-4f95-9cc5-730605c1337e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:40.420263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 380/740] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:40 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:19:40.421398 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60996dae-74ea-4216-9b8a-f0c216b4fa08 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:40.525467 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:40.551550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf91636-a02f-4f95-9cc5-730605c1337e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:40.565496 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:40.576367 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60996dae-74ea-4216-9b8a-f0c216b4fa08 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:40.608234 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35e47d-c068-4aa0-93e3-3b139319b34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:40.610176 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35e47d-c068-4aa0-93e3-3b139319b34c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:40.610740 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35e47d-c068-4aa0-93e3-3b139319b34c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:40.611487 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35e47d-c068-4aa0-93e3-3b139319b34c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:40.623184 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a75e4-2591-484f-8cdf-f65b568e35ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:40.623905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a75e4-2591-484f-8cdf-f65b568e35ba None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:40.624211 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a75e4-2591-484f-8cdf-f65b568e35ba None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:40.624493 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a75e4-2591-484f-8cdf-f65b568e35ba None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:40.625698 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a75e4-2591-484f-8cdf-f65b568e35ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:40.625698 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a75e4-2591-484f-8cdf-f65b568e35ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:40.627052 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a75e4-2591-484f-8cdf-f65b568e35ba None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:40.631192 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc35e47d-c068-4aa0-93e3-3b139319b34c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:40.633123 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc35e47d-c068-4aa0-93e3-3b139319b34c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:40.646081 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 381/741] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:40 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:40.646081 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a75e4-2591-484f-8cdf-f65b568e35ba None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:40.662013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-dc35e47d-c068-4aa0-93e3-3b139319b34c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:40.662013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:40.662013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:40.662013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:40.662013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:40.662013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:40.662013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:40.662013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:40.662013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:40.662013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:40.662013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:40.662013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:40.662013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:40.662013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:40.662013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:40.663104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:40.663104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:40.663104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:40.663104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:40.663104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:40.663104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:40.663104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:40.663104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:40.663104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:40.663104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:40.663104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:40.663104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:40.663104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:40.663104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:40.663104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:40.663104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:40.663104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:40.664073 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:40.664073 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:40.664073 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:40.664073 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:40.665973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 361/742] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:40.671879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35e47d-c068-4aa0-93e3-3b139319b34c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:40.772816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a75e4-2591-484f-8cdf-f65b568e35ba None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:40.808637 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:40.815368 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9cc5e8-8ec3-47ab-81ab-cbe55759e219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:40.816047 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cc5e8-8ec3-47ab-81ab-cbe55759e219 None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:40.816269 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35e47d-c068-4aa0-93e3-3b139319b34c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:40.816413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cc5e8-8ec3-47ab-81ab-cbe55759e219 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:40.816653 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cc5e8-8ec3-47ab-81ab-cbe55759e219 None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:40.817300 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9cc5e8-8ec3-47ab-81ab-cbe55759e219 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:40.818744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9cc5e8-8ec3-47ab-81ab-cbe55759e219 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:40.847088 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 382/743] 10.222.0.121 () {60 vars in 1146 bytes} [Thu Aug 4 10:19:40 2022] POST /identity/v3/services => generated 252 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:19:40.847088 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9cc5e8-8ec3-47ab-81ab-cbe55759e219 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:40.894581 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f3331-b583-4842-977e-e89f34bf36c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:40.896578 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f3331-b583-4842-977e-e89f34bf36c1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:40.897134 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f3331-b583-4842-977e-e89f34bf36c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:40.897587 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f3331-b583-4842-977e-e89f34bf36c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:40.908411 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f3331-b583-4842-977e-e89f34bf36c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:40.910453 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f3331-b583-4842-977e-e89f34bf36c1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:40.925216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-ce0f3331-b583-4842-977e-e89f34bf36c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:40.925216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:40.925216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:40.925216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:40.925216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:40.925216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:40.925216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:40.925216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:40.925216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:40.925216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:40.925216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:40.925216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:40.925216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:19:40.925216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:19:40.925216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:19:40.926366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:19:40.926366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:40.926366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:40.926366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:40.926366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:40.926366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:40.926366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:40.926366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:40.926366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:40.926366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:40.926366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:40.926366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:40.926366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:40.926366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:40.926366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:40.926366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:40.926366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:40.927455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:19:40.927455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:19:40.927455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:19:40.927455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:19:40.927455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:19:40.927455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:19:40.927455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:40.927455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:40.927455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 362/744] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:40.928133 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ad364-b621-4ef0-ada7-4f19f1b5c557 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:40.928729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ad364-b621-4ef0-ada7-4f19f1b5c557 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:40.928729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ad364-b621-4ef0-ada7-4f19f1b5c557 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:40.935278 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 363/745] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:40 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:40.935278 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1cd98c-3184-427f-a7fa-abb4cfec59b8 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:40.935278 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1cd98c-3184-427f-a7fa-abb4cfec59b8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:40.935278 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1cd98c-3184-427f-a7fa-abb4cfec59b8 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:40.935278 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 364/746] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:40.935278 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1cd98c-3184-427f-a7fa-abb4cfec59b8 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:40.958651 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:40.986696 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9cc5e8-8ec3-47ab-81ab-cbe55759e219 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:41.037189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a53d294-ccd8-45ba-ba6b-687dd7da1c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:41.038269 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53d294-ccd8-45ba-ba6b-687dd7da1c05 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:41.038575 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53d294-ccd8-45ba-ba6b-687dd7da1c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:41.038853 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53d294-ccd8-45ba-ba6b-687dd7da1c05 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:41.039704 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a53d294-ccd8-45ba-ba6b-687dd7da1c05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:41.039868 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a53d294-ccd8-45ba-ba6b-687dd7da1c05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:41.041529 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a53d294-ccd8-45ba-ba6b-687dd7da1c05 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:41.054961 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 383/747] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:40 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:41.057162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f13d20-1745-4fde-ba09-52786468b5f8 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:41.057456 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f13d20-1745-4fde-ba09-52786468b5f8 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:41.057735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f13d20-1745-4fde-ba09-52786468b5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:41.065325 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1cd98c-3184-427f-a7fa-abb4cfec59b8 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:41.079908 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-d4f13d20-1745-4fde-ba09-52786468b5f8 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:41.081181 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d4f13d20-1745-4fde-ba09-52786468b5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:41.109243 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c29f5f3-aa17-4221-bdb5-11599119398a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:41.109956 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29f5f3-aa17-4221-bdb5-11599119398a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:41.110409 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29f5f3-aa17-4221-bdb5-11599119398a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:41.110701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29f5f3-aa17-4221-bdb5-11599119398a None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:41.112148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c29f5f3-aa17-4221-bdb5-11599119398a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:41.112476 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c29f5f3-aa17-4221-bdb5-11599119398a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:41.114919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c29f5f3-aa17-4221-bdb5-11599119398a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:41.128958 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 365/748] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:19:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:41.130280 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98c8e5-671c-4dad-b639-1028c20d2362 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:41.130570 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98c8e5-671c-4dad-b639-1028c20d2362 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:41.130871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98c8e5-671c-4dad-b639-1028c20d2362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:41.144806 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:41.153396 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-3e98c8e5-671c-4dad-b639-1028c20d2362 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:41.154680 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3e98c8e5-671c-4dad-b639-1028c20d2362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:41.173482 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 384/749] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:41 2022] POST /identity/v3/auth/tokens => generated 904 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:19:41.177191 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f13d20-1745-4fde-ba09-52786468b5f8 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:41.245164 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 366/750] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:41 2022] POST /identity/v3/auth/tokens => generated 904 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 10:19:41.245973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98c8e5-671c-4dad-b639-1028c20d2362 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:41.257895 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:41.308843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f13d20-1745-4fde-ba09-52786468b5f8 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:41.348523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b426789-4776-42f0-b2f6-c82b3999ce14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:41.349293 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b426789-4776-42f0-b2f6-c82b3999ce14 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:41.349642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b426789-4776-42f0-b2f6-c82b3999ce14 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:41.349925 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b426789-4776-42f0-b2f6-c82b3999ce14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:41.358318 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b426789-4776-42f0-b2f6-c82b3999ce14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:41.359839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b426789-4776-42f0-b2f6-c82b3999ce14 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:41.363730 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98c8e5-671c-4dad-b639-1028c20d2362 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:41.374896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-7b426789-4776-42f0-b2f6-c82b3999ce14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:41.374896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:41.374896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:41.374896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:41.374896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:41.374896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:41.374896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:41.374896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:41.374896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:41.374896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:41.374896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:41.374896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:41.374896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:19:41.374896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:19:41.374896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:19:41.376069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:19:41.376069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:41.376069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:41.376069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:41.376069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:41.376069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:41.376069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:41.376069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:41.376069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:41.376069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:41.376069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:41.376069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:41.376069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:41.376069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:41.376069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:41.376069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:41.376069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:41.377074 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:19:41.377074 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:19:41.377074 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:19:41.377074 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:19:41.377074 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:19:41.377074 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:19:41.377074 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:41.377074 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:41.381193 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 385/751] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:41.382123 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b426789-4776-42f0-b2f6-c82b3999ce14 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:41.408378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916905d-1174-4add-9cf4-24ec4be9f2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:41.409288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916905d-1174-4add-9cf4-24ec4be9f2f1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:41.409724 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916905d-1174-4add-9cf4-24ec4be9f2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:41.410008 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916905d-1174-4add-9cf4-24ec4be9f2f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:41.410830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916905d-1174-4add-9cf4-24ec4be9f2f1 None admin] RBAC: Adding query filter params (role.id=2edb704d42834db6acd295fa3a8be5d5, scope.project.id=22768aba760b4307aab131d1312b2bf6, user.id=efe6e59228bd49bfbff25a11eb682426) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:41.411157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916905d-1174-4add-9cf4-24ec4be9f2f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:41.412797 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916905d-1174-4add-9cf4-24ec4be9f2f1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:41.419569 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 367/752] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:19:41 2022] GET /identity/v3/role_assignments?user.id=efe6e59228bd49bfbff25a11eb682426&scope.project.id=22768aba760b4307aab131d1312b2bf6&role.id=2edb704d42834db6acd295fa3a8be5d5 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:41.420764 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916905d-1174-4add-9cf4-24ec4be9f2f1 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:41.434743 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:41.539320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b426789-4776-42f0-b2f6-c82b3999ce14 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:41.549616 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:41.624914 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916905d-1174-4add-9cf4-24ec4be9f2f1 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:41.625894 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3d5c3-6281-43e0-8871-26ff75a06913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:41.627427 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3d5c3-6281-43e0-8871-26ff75a06913 None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:41.629671 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3d5c3-6281-43e0-8871-26ff75a06913 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:41.630629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3d5c3-6281-43e0-8871-26ff75a06913 None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:41.631653 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3d5c3-6281-43e0-8871-26ff75a06913 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:41.633499 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3d5c3-6281-43e0-8871-26ff75a06913 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:41.664933 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 386/753] 10.222.0.121 () {60 vars in 1146 bytes} [Thu Aug 4 10:19:41 2022] POST /identity/v3/services => generated 241 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:19:41.665590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3d5c3-6281-43e0-8871-26ff75a06913 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:41.702760 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b903b33-3889-4306-966e-67df4d15f144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:41.703347 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b903b33-3889-4306-966e-67df4d15f144 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:41.703695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b903b33-3889-4306-966e-67df4d15f144 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:41.705530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b903b33-3889-4306-966e-67df4d15f144 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:41.725879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b903b33-3889-4306-966e-67df4d15f144 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:41.732212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-5b903b33-3889-4306-966e-67df4d15f144 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:19:41.732212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:41.732212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:41.732212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:41.732212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:41.732212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:41.732212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:41.732212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:41.732212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:41.732212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:41.732212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:41.732212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:41.732212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:41.732212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:41.732212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:41.732212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:41.734189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:41.734189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:41.734189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:41.734189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:41.734189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:41.734189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:41.734189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:41.734189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:41.734189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:41.734189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:41.734189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:41.734189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:41.734189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:41.734189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:41.734189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:41.734189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:41.735998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:41.735998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:41.735998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:41.735998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:41.735998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:41.735998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:41.735998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:41.735998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:41.735998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:41.735998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:41.735998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:19:41.735998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:41.735998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 368/754] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:19:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:41.740969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b903b33-3889-4306-966e-67df4d15f144 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:41.893254 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3d5c3-6281-43e0-8871-26ff75a06913 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:41.896628 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:41.946911 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d93dd4-5af4-41a2-a359-24531c5e1570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:41.947584 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d93dd4-5af4-41a2-a359-24531c5e1570 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:41.948007 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d93dd4-5af4-41a2-a359-24531c5e1570 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:41.948296 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d93dd4-5af4-41a2-a359-24531c5e1570 None admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:41.949087 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d93dd4-5af4-41a2-a359-24531c5e1570 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:41.949405 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d93dd4-5af4-41a2-a359-24531c5e1570 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:41.951005 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d93dd4-5af4-41a2-a359-24531c5e1570 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:41.959067 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b903b33-3889-4306-966e-67df4d15f144 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:41.966627 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 387/755] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:19:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:41.973114 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d93dd4-5af4-41a2-a359-24531c5e1570 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:41.992000 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:42.000697 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc9eee-8845-4ac8-9679-1f85312aa4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:42.001356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc9eee-8845-4ac8-9679-1f85312aa4aa None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:42.001645 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc9eee-8845-4ac8-9679-1f85312aa4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:42.001927 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc9eee-8845-4ac8-9679-1f85312aa4aa None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:42.002694 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc9eee-8845-4ac8-9679-1f85312aa4aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:42.003009 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc9eee-8845-4ac8-9679-1f85312aa4aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:42.004567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc9eee-8845-4ac8-9679-1f85312aa4aa None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:42.015052 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 369/756] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:19:41 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:42.021425 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc9eee-8845-4ac8-9679-1f85312aa4aa None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:42.135389 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d93dd4-5af4-41a2-a359-24531c5e1570 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:42.179377 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc9eee-8845-4ac8-9679-1f85312aa4aa None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:42.197146 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732a840-693f-49e8-8ae8-e781b04b5e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:42.198038 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732a840-693f-49e8-8ae8-e781b04b5e63 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:42.198330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732a840-693f-49e8-8ae8-e781b04b5e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:42.198613 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732a840-693f-49e8-8ae8-e781b04b5e63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:42.199542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732a840-693f-49e8-8ae8-e781b04b5e63 None admin] RBAC: Adding query filter params (role.id=7342106a255b4bce8f150726602a8b95, scope.project.id=22768aba760b4307aab131d1312b2bf6, user.id=e16e26784601425c9883acd91f65becc) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:42.199898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732a840-693f-49e8-8ae8-e781b04b5e63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:42.211223 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732a840-693f-49e8-8ae8-e781b04b5e63 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:42.231496 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 388/757] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:19:41 2022] GET /identity/v3/role_assignments?user.id=e16e26784601425c9883acd91f65becc&scope.project.id=22768aba760b4307aab131d1312b2bf6&role.id=7342106a255b4bce8f150726602a8b95 => generated 260 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:42.296269 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0fd55-6399-4413-9af2-cc0166513f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:42.297435 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0fd55-6399-4413-9af2-cc0166513f88 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:42.297963 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0fd55-6399-4413-9af2-cc0166513f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:42.298453 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0fd55-6399-4413-9af2-cc0166513f88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:42.321184 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0fd55-6399-4413-9af2-cc0166513f88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:42.323855 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0fd55-6399-4413-9af2-cc0166513f88 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:42.345938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-e9a0fd55-6399-4413-9af2-cc0166513f88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:42.345938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:42.345938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:42.345938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:42.345938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:42.345938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:42.345938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:42.345938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:42.345938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:42.345938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:42.345938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:42.345938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:42.345938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:42.345938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:42.345938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:42.347002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:42.347002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:42.347002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:42.347002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:42.347002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:42.347002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:42.347002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:42.347002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:42.347002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:42.347002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:42.347002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:42.347002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:42.347002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:42.347002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:42.347002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:42.347002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:42.347002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:42.352382 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:42.352382 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:42.352382 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:42.352382 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:42.354835 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 370/758] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:19:42.360678 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732a840-693f-49e8-8ae8-e781b04b5e63 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:42.430864 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:42.574676 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732a840-693f-49e8-8ae8-e781b04b5e63 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:42.629084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f5ac9-adc3-4899-a36d-bbc0642d1819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:42.629817 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f5ac9-adc3-4899-a36d-bbc0642d1819 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:42.630115 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f5ac9-adc3-4899-a36d-bbc0642d1819 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:42.630358 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f5ac9-adc3-4899-a36d-bbc0642d1819 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:42.631276 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f5ac9-adc3-4899-a36d-bbc0642d1819 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:42.631582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f5ac9-adc3-4899-a36d-bbc0642d1819 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:42.633814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f5ac9-adc3-4899-a36d-bbc0642d1819 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:42.655929 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 389/759] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:42 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:42.661300 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0fd55-6399-4413-9af2-cc0166513f88 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:42.819453 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:42.865810 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0fd55-6399-4413-9af2-cc0166513f88 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:42.933270 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f31091-b6fb-4e03-899c-326999a7aaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:42.934490 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f31091-b6fb-4e03-899c-326999a7aaef None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:42.934982 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f31091-b6fb-4e03-899c-326999a7aaef None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:42.935450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f31091-b6fb-4e03-899c-326999a7aaef None admin] PATH_INFO: `/v3/users/nova` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:42.951716 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f31091-b6fb-4e03-899c-326999a7aaef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:42.954692 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f31091-b6fb-4e03-899c-326999a7aaef None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:42.977843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-c5f31091-b6fb-4e03-899c-326999a7aaef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:19:42.977843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:42.977843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:42.977843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:42.977843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:42.977843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:42.977843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:42.977843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:42.977843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:42.977843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:42.977843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:42.977843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:42.977843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:19:42.977843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:19:42.977843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:19:42.977843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:19:42.978978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:42.978978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:42.978978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:19:42.978978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:42.978978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:19:42.978978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:42.978978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:42.978978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:42.978978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:42.978978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:42.978978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:42.978978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:42.978978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:42.978978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:42.978978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:42.978978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:42.978978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:42.978978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:42.980016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:42.980016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:42.980016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:19:42.980016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:19:42.980016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:19:42.980016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:19:42.980016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:19:42.980016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:19:42.980016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:19:42.980016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:42.980603 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 371/760] 10.222.0.121 () {56 vars in 1098 bytes} [Thu Aug 4 10:19:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:42.987316 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f5ac9-adc3-4899-a36d-bbc0642d1819 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:43.096535 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:43.112189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fe450-df87-4157-90e1-6ffbc508a489 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:43.112984 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fe450-df87-4157-90e1-6ffbc508a489 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:43.113166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fe450-df87-4157-90e1-6ffbc508a489 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:43.114889 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 372/761] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:43 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:43.121543 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f5ac9-adc3-4899-a36d-bbc0642d1819 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:43.130002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224266a7-db47-4f64-86cd-9e614d486e05 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:43.130379 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224266a7-db47-4f64-86cd-9e614d486e05 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:43.130554 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224266a7-db47-4f64-86cd-9e614d486e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:43.162487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251f79b-bfa4-48d8-bb8d-e92626c6b710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:43.163138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251f79b-bfa4-48d8-bb8d-e92626c6b710 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:43.163419 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251f79b-bfa4-48d8-bb8d-e92626c6b710 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:43.163710 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251f79b-bfa4-48d8-bb8d-e92626c6b710 None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:43.164661 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1251f79b-bfa4-48d8-bb8d-e92626c6b710 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:43.164978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1251f79b-bfa4-48d8-bb8d-e92626c6b710 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:43.166661 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1251f79b-bfa4-48d8-bb8d-e92626c6b710 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:43.169149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-224266a7-db47-4f64-86cd-9e614d486e05 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:43.170558 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-224266a7-db47-4f64-86cd-9e614d486e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:43.187821 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 390/762] 10.222.0.121 () {56 vars in 1138 bytes} [Thu Aug 4 10:19:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:43.197489 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251f79b-bfa4-48d8-bb8d-e92626c6b710 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:43.288149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 373/763] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:43 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:19:43.307663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224266a7-db47-4f64-86cd-9e614d486e05 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:43.343621 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:43.425137 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224266a7-db47-4f64-86cd-9e614d486e05 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:43.446289 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251f79b-bfa4-48d8-bb8d-e92626c6b710 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:43.466345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff0a9a-4032-4ff6-84b0-dbdc341fa0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:43.467027 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff0a9a-4032-4ff6-84b0-dbdc341fa0be None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:43.467315 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff0a9a-4032-4ff6-84b0-dbdc341fa0be None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:43.467594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff0a9a-4032-4ff6-84b0-dbdc341fa0be None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:43.468310 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff0a9a-4032-4ff6-84b0-dbdc341fa0be None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:43.469724 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff0a9a-4032-4ff6-84b0-dbdc341fa0be None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:43.482174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-34ff0a9a-4032-4ff6-84b0-dbdc341fa0be None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 10:19:43.482174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:43.482174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:19:43.482174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:19:43.482174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:19:43.482174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:19:43.482174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:19:43.482174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:19:43.482174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 10:19:43.482174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:43.482174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:19:43.482174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:43.482174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:43.482174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:43.482174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:43.482174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:43.483313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:43.483313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:43.483313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:43.483313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:43.483313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:43.483313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:43.483313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:43.483313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:19:43.483313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:19:43.483313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:19:43.483313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:19:43.483313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:43.483313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:43.483313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:43.483313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:43.483313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:43.484258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:43.484258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:43.484258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:43.484258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:43.484258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:43.484258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:43.484258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:43.484258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:43.484258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:43.484258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:43.484258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:43.484258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:19:43.484258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:19:43.484258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 10:19:43.484258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:43.488124 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 374/764] 10.222.0.121 () {56 vars in 1134 bytes} [Thu Aug 4 10:19:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:43.490951 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff0a9a-4032-4ff6-84b0-dbdc341fa0be None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:43.511349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9d42e-6860-4592-8bb7-a9d0cd9b8442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:43.512545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9d42e-6860-4592-8bb7-a9d0cd9b8442 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:43.513061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9d42e-6860-4592-8bb7-a9d0cd9b8442 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:43.513564 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9d42e-6860-4592-8bb7-a9d0cd9b8442 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:43.525423 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad9d42e-6860-4592-8bb7-a9d0cd9b8442 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:43.527432 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad9d42e-6860-4592-8bb7-a9d0cd9b8442 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:43.540339 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:43.563297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-aad9d42e-6860-4592-8bb7-a9d0cd9b8442 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:43.563297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:43.563297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:43.563297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:43.563297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:43.563297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:43.563297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:43.563297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:43.563297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:43.563297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:43.563297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:43.563297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:43.563297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:43.563297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:43.563297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:43.564369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:43.564369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:43.564369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:43.564369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:43.564369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:43.564369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:43.564369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:43.564369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:43.564369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:43.564369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:43.564369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:43.564369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:43.564369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:43.564369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:43.564369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:43.564369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:43.564369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:43.565317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:43.565317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:43.565317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:43.565317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:43.572655 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 391/765] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:19:43.578761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9d42e-6860-4592-8bb7-a9d0cd9b8442 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:43.621725 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:43.630040 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff0a9a-4032-4ff6-84b0-dbdc341fa0be None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:43.668859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd1d5c-3767-4c22-b0f8-358c0b0267e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:43.669657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd1d5c-3767-4c22-b0f8-358c0b0267e0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:43.669963 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd1d5c-3767-4c22-b0f8-358c0b0267e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:43.670305 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd1d5c-3767-4c22-b0f8-358c0b0267e0 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:43.671085 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd1d5c-3767-4c22-b0f8-358c0b0267e0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:43.671466 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd1d5c-3767-4c22-b0f8-358c0b0267e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:43.673039 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd1d5c-3767-4c22-b0f8-358c0b0267e0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:43.683073 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 375/766] 10.222.0.121 () {56 vars in 1129 bytes} [Thu Aug 4 10:19:43 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:19:43.684980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd1d5c-3767-4c22-b0f8-358c0b0267e0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:43.797351 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd1d5c-3767-4c22-b0f8-358c0b0267e0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:43.836465 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a2737-31a0-4697-a549-2cd3a98c47c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:43.837422 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a2737-31a0-4697-a549-2cd3a98c47c3 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:43.837912 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a2737-31a0-4697-a549-2cd3a98c47c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:43.838339 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a2737-31a0-4697-a549-2cd3a98c47c3 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:43.839160 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a2737-31a0-4697-a549-2cd3a98c47c3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:43.839586 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a2737-31a0-4697-a549-2cd3a98c47c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:43.843976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a2737-31a0-4697-a549-2cd3a98c47c3 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:43.847268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9d42e-6860-4592-8bb7-a9d0cd9b8442 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:43.852170 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 376/767] 10.222.0.121 () {56 vars in 1129 bytes} [Thu Aug 4 10:19:43 2022] GET /identity/v3/services?type=block-storage => generated 369 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:19:43.855253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a2737-31a0-4697-a549-2cd3a98c47c3 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:43.957204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f1535-9012-487d-9a21-97f782690d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:43.957780 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f1535-9012-487d-9a21-97f782690d1a None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:43.958072 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f1535-9012-487d-9a21-97f782690d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:43.958354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f1535-9012-487d-9a21-97f782690d1a None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:43.959226 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f1535-9012-487d-9a21-97f782690d1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:43.959544 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f1535-9012-487d-9a21-97f782690d1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:43.961285 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f1535-9012-487d-9a21-97f782690d1a None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:43.988650 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 392/768] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:43.994299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f1535-9012-487d-9a21-97f782690d1a None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:44.032484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:44.165107 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:44.213789 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a2737-31a0-4697-a549-2cd3a98c47c3 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:44.227790 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f1535-9012-487d-9a21-97f782690d1a None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:44.273189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd95b1-72c2-4dd6-b8fa-3da333e9f1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:44.274623 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd95b1-72c2-4dd6-b8fa-3da333e9f1cf None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:44.275123 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd95b1-72c2-4dd6-b8fa-3da333e9f1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:44.275609 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd95b1-72c2-4dd6-b8fa-3da333e9f1cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:44.280953 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740bc43-ae13-4f86-9911-bcb529532ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:44.281957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740bc43-ae13-4f86-9911-bcb529532ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:44.282401 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740bc43-ae13-4f86-9911-bcb529532ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:44.282859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740bc43-ae13-4f86-9911-bcb529532ae4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:44.284380 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740bc43-ae13-4f86-9911-bcb529532ae4 None admin] RBAC: Adding query filter params (interface=public, service_id=3d0ac2d734184011975ad132ee5832ec, region_id=RegionOne) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:44.285000 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740bc43-ae13-4f86-9911-bcb529532ae4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:44.285340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd95b1-72c2-4dd6-b8fa-3da333e9f1cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:44.287186 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd95b1-72c2-4dd6-b8fa-3da333e9f1cf None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:44.287667 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740bc43-ae13-4f86-9911-bcb529532ae4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:44.302407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-a8bd95b1-72c2-4dd6-b8fa-3da333e9f1cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:44.302407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:44.302407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:44.302407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:44.302407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:44.302407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:44.302407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:44.302407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:44.302407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:44.302407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:44.302407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:44.302407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:44.302407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:19:44.302407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:19:44.302407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:19:44.303501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:19:44.303501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:44.303501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:44.303501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:44.303501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:44.303501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:44.303501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:44.303501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:44.303501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:44.303501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:44.303501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:44.303501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:44.303501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:44.303501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:44.303501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:44.303501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:44.303501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:44.304440 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:19:44.304440 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:19:44.304440 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:19:44.304440 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:19:44.304440 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:19:44.304440 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:19:44.304440 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:44.304440 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:44.308022 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 393/770] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:19:44.308159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 377/770] 10.222.0.121 () {56 vars in 1256 bytes} [Thu Aug 4 10:19:43 2022] GET /identity/v3/endpoints?service_id=3d0ac2d734184011975ad132ee5832ec&interface=public®ion_id=RegionOne => generated 195 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:19:44.310919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf28cf-3ea5-4540-9f29-30795325937a None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:44.311400 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf28cf-3ea5-4540-9f29-30795325937a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:44.312110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf28cf-3ea5-4540-9f29-30795325937a None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:44.313466 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd95b1-72c2-4dd6-b8fa-3da333e9f1cf None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:44.336270 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 378/771] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:44 2022] GET /identity => generated 271 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:44.337930 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf28cf-3ea5-4540-9f29-30795325937a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:44.466886 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd95b1-72c2-4dd6-b8fa-3da333e9f1cf None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:44.490991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:44.500436 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:44.520538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b530f7c-d58c-4c64-a00b-29200ad3efb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:44.521565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b530f7c-d58c-4c64-a00b-29200ad3efb8 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:44.522064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b530f7c-d58c-4c64-a00b-29200ad3efb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:44.522537 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b530f7c-d58c-4c64-a00b-29200ad3efb8 None admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:44.522910 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf28cf-3ea5-4540-9f29-30795325937a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:44.523984 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b530f7c-d58c-4c64-a00b-29200ad3efb8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:44.524556 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b530f7c-d58c-4c64-a00b-29200ad3efb8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:44.529418 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b530f7c-d58c-4c64-a00b-29200ad3efb8 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:44.562461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 394/772] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:19:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:19:44.567647 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfde973-1708-4752-a65f-3a079a35c8bb None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:44.567647 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfde973-1708-4752-a65f-3a079a35c8bb None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:44.567647 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfde973-1708-4752-a65f-3a079a35c8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:44.596599 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-0dfde973-1708-4752-a65f-3a079a35c8bb None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:44.598795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-0dfde973-1708-4752-a65f-3a079a35c8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:44.599565 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02179f57-2860-4695-a9ca-593301f70464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:44.600713 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02179f57-2860-4695-a9ca-593301f70464 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:44.601208 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02179f57-2860-4695-a9ca-593301f70464 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:44.601695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02179f57-2860-4695-a9ca-593301f70464 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:44.602901 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02179f57-2860-4695-a9ca-593301f70464 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:44.605603 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02179f57-2860-4695-a9ca-593301f70464 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:44.616746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 379/773] 10.222.0.121 () {56 vars in 1092 bytes} [Thu Aug 4 10:19:44 2022] GET /identity/v3/services => generated 776 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:44.617753 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02179f57-2860-4695-a9ca-593301f70464 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:44.733880 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 395/774] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:44 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:19:44.735701 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699dd923-e4dc-432a-9fed-f7f4187d0f0e None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:44.736019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699dd923-e4dc-432a-9fed-f7f4187d0f0e None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:44.736304 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699dd923-e4dc-432a-9fed-f7f4187d0f0e None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:44.739632 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 396/775] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:44 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:44.746581 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eff406-7afb-48fa-908e-7d5d0cdaf512 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:44.746921 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eff406-7afb-48fa-908e-7d5d0cdaf512 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:44.747311 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eff406-7afb-48fa-908e-7d5d0cdaf512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:44.781660 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-97eff406-7afb-48fa-908e-7d5d0cdaf512 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:44.782610 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-97eff406-7afb-48fa-908e-7d5d0cdaf512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:44.800571 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02179f57-2860-4695-a9ca-593301f70464 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:44.842059 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e142f4-3e01-4586-a9bd-2e485dff1427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:44.842761 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e142f4-3e01-4586-a9bd-2e485dff1427 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:44.843040 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e142f4-3e01-4586-a9bd-2e485dff1427 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:44.843315 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e142f4-3e01-4586-a9bd-2e485dff1427 None admin] PATH_INFO: `/v3/projects/22768aba760b4307aab131d1312b2bf6/users/0d947aeb749546309a8d4b7938087ba0/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:44.845345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e142f4-3e01-4586-a9bd-2e485dff1427 None admin] RBAC: Authorizing `identity:create_grant(project_id=22768aba760b4307aab131d1312b2bf6, user_id=0d947aeb749546309a8d4b7938087ba0, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:44.846824 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e142f4-3e01-4586-a9bd-2e485dff1427 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:44.863412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 397/776] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:44 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:19:44.863877 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 380/777] 10.222.0.121 () {58 vars in 1444 bytes} [Thu Aug 4 10:19:44 2022] PUT /identity/v3/projects/22768aba760b4307aab131d1312b2bf6/users/0d947aeb749546309a8d4b7938087ba0/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:19:44.864791 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e142f4-3e01-4586-a9bd-2e485dff1427 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:44.865803 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93ae96-8f42-4095-acd1-d992bc2cbb4a None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:44.866216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93ae96-8f42-4095-acd1-d992bc2cbb4a None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:44.866507 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93ae96-8f42-4095-acd1-d992bc2cbb4a None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:44.867911 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 398/778] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:44.875072 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be5c27-3d30-49db-a00f-098f9e470b83 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:44.875072 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be5c27-3d30-49db-a00f-098f9e470b83 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:44.875072 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be5c27-3d30-49db-a00f-098f9e470b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:44.892602 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:44.895502 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-53be5c27-3d30-49db-a00f-098f9e470b83 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:44.896980 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-53be5c27-3d30-49db-a00f-098f9e470b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:44.939822 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:45.010598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 399/779] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:44 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:19:45.012245 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be5c27-3d30-49db-a00f-098f9e470b83 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:45.075214 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e142f4-3e01-4586-a9bd-2e485dff1427 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:45.131586 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf017a-45f1-4a65-8383-24a60b0bc08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:45.132771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf017a-45f1-4a65-8383-24a60b0bc08e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:45.133286 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf017a-45f1-4a65-8383-24a60b0bc08e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:45.133785 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf017a-45f1-4a65-8383-24a60b0bc08e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:45.146772 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf017a-45f1-4a65-8383-24a60b0bc08e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:45.152383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-09bf017a-45f1-4a65-8383-24a60b0bc08e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:45.152383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:45.152383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:45.152383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:45.152383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:45.152383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:45.152383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:45.152383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:45.152383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:45.152383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:45.152383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:45.152383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:45.152383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:45.152383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:45.152383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:45.152383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:45.153523 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:45.153523 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:45.153523 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:45.153523 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:45.153523 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:45.153523 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:45.153523 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:45.153523 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:45.153523 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:45.153523 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:45.153523 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:45.153523 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:45.153523 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:45.153523 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:45.153523 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:45.153523 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:45.154479 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:45.154479 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:45.154479 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:45.154479 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:45.154479 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:45.154479 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:45.154479 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:45.154479 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:45.154479 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:45.154479 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:45.154479 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:45.154479 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:45.158514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 381/780] 10.222.0.121 () {56 vars in 1107 bytes} [Thu Aug 4 10:19:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:45.159704 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf017a-45f1-4a65-8383-24a60b0bc08e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:45.176101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be5c27-3d30-49db-a00f-098f9e470b83 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:45.214855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa6e34-c26e-4e2e-bd6c-49e6e16d89e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:45.215543 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa6e34-c26e-4e2e-bd6c-49e6e16d89e1 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:45.215959 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa6e34-c26e-4e2e-bd6c-49e6e16d89e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:45.216165 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa6e34-c26e-4e2e-bd6c-49e6e16d89e1 None admin] PATH_INFO: `/v3/services/network` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:45.216801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa6e34-c26e-4e2e-bd6c-49e6e16d89e1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:45.218383 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa6e34-c26e-4e2e-bd6c-49e6e16d89e1 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:45.228659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-a5aa6e34-c26e-4e2e-bd6c-49e6e16d89e1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 10:19:45.228659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:45.228659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:19:45.228659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:19:45.228659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:19:45.228659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:19:45.228659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:19:45.228659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:19:45.228659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 10:19:45.228659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:45.228659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:19:45.228659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:45.228659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:45.228659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:45.228659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:45.228659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:45.228659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:45.229843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:45.229843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:45.229843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:45.229843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:45.229843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:45.229843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:45.229843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:19:45.229843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:19:45.229843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:19:45.229843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:19:45.229843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:45.229843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:45.229843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:45.229843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:45.229843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:45.229843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:45.230736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:45.230736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:45.230736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:45.230736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:45.230736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:45.230736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:45.230736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:45.230736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:45.230736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:45.230736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:45.230736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:19:45.230736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:19:45.230736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 10:19:45.230736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:45.230736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 400/781] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:45 2022] GET /identity/v3/services/network => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:45.231569 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa6e34-c26e-4e2e-bd6c-49e6e16d89e1 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:45.250240 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:45.269111 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:45.288689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf017a-45f1-4a65-8383-24a60b0bc08e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:45.342760 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c247386-b03a-4161-933c-94714a94bc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:45.344630 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c247386-b03a-4161-933c-94714a94bc18 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:45.345466 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c247386-b03a-4161-933c-94714a94bc18 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:45.346256 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c247386-b03a-4161-933c-94714a94bc18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:45.367559 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c247386-b03a-4161-933c-94714a94bc18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:45.378853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-4c247386-b03a-4161-933c-94714a94bc18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:19:45.378853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:45.378853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:45.378853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:45.378853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:45.378853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:45.378853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:45.378853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:45.378853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:45.378853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:45.378853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:45.378853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:45.378853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:45.378853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:45.378853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:45.378853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:45.380848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:45.380848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:45.380848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:45.380848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:45.380848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:45.380848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:45.380848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:45.380848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:45.380848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:45.380848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:45.380848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:45.380848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:45.380848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:45.380848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:45.380848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:45.380848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:45.382579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:45.382579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:45.382579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:45.382579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:45.382579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:45.382579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:45.382579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:45.382579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:45.382579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:45.382579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:45.382579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:19:45.382579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:45.389852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 382/782] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:19:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:45.391246 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa6e34-c26e-4e2e-bd6c-49e6e16d89e1 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:45.392277 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c247386-b03a-4161-933c-94714a94bc18 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:45.453373 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962afc7-da9d-4f47-8e29-e14421417258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:45.454397 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962afc7-da9d-4f47-8e29-e14421417258 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:45.454874 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962afc7-da9d-4f47-8e29-e14421417258 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:45.455415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962afc7-da9d-4f47-8e29-e14421417258 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:45.456702 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962afc7-da9d-4f47-8e29-e14421417258 None admin] RBAC: Adding query filter params (name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:45.457225 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962afc7-da9d-4f47-8e29-e14421417258 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:45.459676 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962afc7-da9d-4f47-8e29-e14421417258 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:45.475863 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 401/783] 10.222.0.121 () {56 vars in 1108 bytes} [Thu Aug 4 10:19:45 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:45.477108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962afc7-da9d-4f47-8e29-e14421417258 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:45.578642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c247386-b03a-4161-933c-94714a94bc18 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:45.647283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a3278-5c67-4186-9a5e-76fc9ef91c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:45.648998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a3278-5c67-4186-9a5e-76fc9ef91c02 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:45.649846 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a3278-5c67-4186-9a5e-76fc9ef91c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:45.650638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a3278-5c67-4186-9a5e-76fc9ef91c02 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:45.652231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:45.654126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a3278-5c67-4186-9a5e-76fc9ef91c02 None admin] RBAC: Adding query filter params (name=network) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:45.655105 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a3278-5c67-4186-9a5e-76fc9ef91c02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:45.659457 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a3278-5c67-4186-9a5e-76fc9ef91c02 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:45.664156 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:45.671295 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 383/784] 10.222.0.121 () {56 vars in 1117 bytes} [Thu Aug 4 10:19:45 2022] GET /identity/v3/services?name=network => generated 125 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:45.673452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a3278-5c67-4186-9a5e-76fc9ef91c02 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:45.682598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962afc7-da9d-4f47-8e29-e14421417258 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:45.744251 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1d8b2-1e95-4bbd-afee-27d4c0698377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:45.745818 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1d8b2-1e95-4bbd-afee-27d4c0698377 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:45.746542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1d8b2-1e95-4bbd-afee-27d4c0698377 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:45.747257 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1d8b2-1e95-4bbd-afee-27d4c0698377 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:45.748810 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1d8b2-1e95-4bbd-afee-27d4c0698377 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:45.749593 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1d8b2-1e95-4bbd-afee-27d4c0698377 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:45.752614 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1d8b2-1e95-4bbd-afee-27d4c0698377 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:45.769171 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 402/785] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:19:45 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:45.771006 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1d8b2-1e95-4bbd-afee-27d4c0698377 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:45.827709 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a3278-5c67-4186-9a5e-76fc9ef91c02 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:45.866854 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45cc762-5dce-44ef-a50b-efa7851c2c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:45.867522 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cc762-5dce-44ef-a50b-efa7851c2c66 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:45.867859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cc762-5dce-44ef-a50b-efa7851c2c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:45.868225 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cc762-5dce-44ef-a50b-efa7851c2c66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:45.880609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45cc762-5dce-44ef-a50b-efa7851c2c66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:45.882388 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45cc762-5dce-44ef-a50b-efa7851c2c66 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:45.898937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-c45cc762-5dce-44ef-a50b-efa7851c2c66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:45.898937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:45.898937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:45.898937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:45.898937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:45.898937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:45.898937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:45.898937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:45.898937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:45.898937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:45.898937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:45.898937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:45.898937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:45.898937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:45.898937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:45.900910 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:45.900910 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:45.900910 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:45.900910 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:45.900910 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:45.900910 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:45.900910 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:45.900910 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:45.900910 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:45.900910 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:45.900910 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:45.900910 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:45.900910 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:45.900910 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:45.900910 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:45.900910 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:45.900910 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:45.902786 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:45.902786 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:45.902786 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:45.902786 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:45.902786 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 384/786] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:45.902786 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45cc762-5dce-44ef-a50b-efa7851c2c66 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:45.963121 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1d8b2-1e95-4bbd-afee-27d4c0698377 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:45.965945 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:46.029123 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acccf469-cf94-499c-9357-f1229d201ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:46.030330 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45cc762-5dce-44ef-a50b-efa7851c2c66 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:46.030875 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acccf469-cf94-499c-9357-f1229d201ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:46.031615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acccf469-cf94-499c-9357-f1229d201ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:46.032347 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acccf469-cf94-499c-9357-f1229d201ad7 None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:46.033823 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acccf469-cf94-499c-9357-f1229d201ad7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:46.034716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acccf469-cf94-499c-9357-f1229d201ad7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:46.037856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acccf469-cf94-499c-9357-f1229d201ad7 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:46.048528 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 403/787] 10.222.0.121 () {56 vars in 1117 bytes} [Thu Aug 4 10:19:45 2022] GET /identity/v3/services?type=network => generated 352 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:46.050659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acccf469-cf94-499c-9357-f1229d201ad7 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:46.072049 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75060099-6bea-49b1-98eb-2811390e67b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:46.072664 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75060099-6bea-49b1-98eb-2811390e67b0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:46.072945 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75060099-6bea-49b1-98eb-2811390e67b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:46.073267 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75060099-6bea-49b1-98eb-2811390e67b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:46.082267 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:46.088955 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75060099-6bea-49b1-98eb-2811390e67b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:46.090561 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75060099-6bea-49b1-98eb-2811390e67b0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:46.107678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-75060099-6bea-49b1-98eb-2811390e67b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:46.107678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:46.107678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:46.107678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:46.107678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:46.107678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:46.107678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:46.107678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:46.107678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:46.107678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:46.107678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:46.107678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:46.107678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:46.107678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:46.107678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:46.108779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:46.108779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:46.108779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:46.108779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:46.108779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:46.108779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:46.108779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:46.108779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:46.108779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:46.108779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:46.108779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:46.108779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:46.108779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:46.108779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:46.108779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:46.108779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:46.108779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:46.109711 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:46.109711 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:46.109711 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:46.109711 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:46.111296 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 385/788] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:46.111424 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75060099-6bea-49b1-98eb-2811390e67b0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:46.178470 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acccf469-cf94-499c-9357-f1229d201ad7 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:46.214960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d739f7f-6d9e-4f4f-ac4a-66fa8ef1aa57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:46.216241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d739f7f-6d9e-4f4f-ac4a-66fa8ef1aa57 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:46.216755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d739f7f-6d9e-4f4f-ac4a-66fa8ef1aa57 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:46.217249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d739f7f-6d9e-4f4f-ac4a-66fa8ef1aa57 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:46.218188 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d739f7f-6d9e-4f4f-ac4a-66fa8ef1aa57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:46.218736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d739f7f-6d9e-4f4f-ac4a-66fa8ef1aa57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:46.220452 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d739f7f-6d9e-4f4f-ac4a-66fa8ef1aa57 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:46.229557 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:46.233482 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75060099-6bea-49b1-98eb-2811390e67b0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:46.235040 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 404/789] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:46 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:46.236418 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d739f7f-6d9e-4f4f-ac4a-66fa8ef1aa57 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:46.274977 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ffc24-7927-410a-a14f-0c6c50560b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:46.275441 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ffc24-7927-410a-a14f-0c6c50560b7e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:46.275724 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ffc24-7927-410a-a14f-0c6c50560b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:46.276214 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ffc24-7927-410a-a14f-0c6c50560b7e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:46.277055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ffc24-7927-410a-a14f-0c6c50560b7e None admin] RBAC: Adding query filter params (interface=public, service_id=fe265f5e54e549a4a1d070139a392a1a, region_id=RegionOne) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:46.277418 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ffc24-7927-410a-a14f-0c6c50560b7e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:46.278819 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ffc24-7927-410a-a14f-0c6c50560b7e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:46.287976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 386/790] 10.222.0.121 () {56 vars in 1256 bytes} [Thu Aug 4 10:19:46 2022] GET /identity/v3/endpoints?service_id=fe265f5e54e549a4a1d070139a392a1a&interface=public®ion_id=RegionOne => generated 195 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:46.287976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ffc24-7927-410a-a14f-0c6c50560b7e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:46.340849 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:46.382038 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d739f7f-6d9e-4f4f-ac4a-66fa8ef1aa57 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:46.425901 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ffc24-7927-410a-a14f-0c6c50560b7e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:46.432885 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfc5b9-7aab-4638-a0f3-d254b4fd4704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:46.433603 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfc5b9-7aab-4638-a0f3-d254b4fd4704 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:46.433965 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfc5b9-7aab-4638-a0f3-d254b4fd4704 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:46.434307 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfc5b9-7aab-4638-a0f3-d254b4fd4704 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:46.435443 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfc5b9-7aab-4638-a0f3-d254b4fd4704 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:46.435822 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfc5b9-7aab-4638-a0f3-d254b4fd4704 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:46.437674 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfc5b9-7aab-4638-a0f3-d254b4fd4704 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:46.453825 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 405/791] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:46 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:46.455167 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfc5b9-7aab-4638-a0f3-d254b4fd4704 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:46.489526 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99b926-3ddb-4565-bf42-ba4dbfc71892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:46.490797 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99b926-3ddb-4565-bf42-ba4dbfc71892 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:46.491349 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99b926-3ddb-4565-bf42-ba4dbfc71892 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:46.491905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99b926-3ddb-4565-bf42-ba4dbfc71892 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:46.506741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99b926-3ddb-4565-bf42-ba4dbfc71892 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:46.509847 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99b926-3ddb-4565-bf42-ba4dbfc71892 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:46.532390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-3d99b926-3ddb-4565-bf42-ba4dbfc71892 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 10:19:46.532390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:46.532390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:46.532390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:46.532390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:46.532390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:46.532390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:46.532390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:46.532390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:46.532390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:46.532390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:46.532390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:46.532390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:19:46.532390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:19:46.532390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:19:46.532390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:19:46.533532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:46.533532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:46.533532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:19:46.533532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:46.533532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:19:46.533532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:46.533532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:46.533532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:46.533532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:46.533532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:46.533532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:46.533532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:46.533532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:46.533532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:46.533532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:46.533532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:46.533532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:46.533532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:46.534551 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:46.534551 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:46.534551 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:19:46.534551 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:19:46.534551 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:19:46.534551 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:19:46.534551 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:19:46.534551 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:19:46.534551 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 10:19:46.534551 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:46.534551 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 387/792] 10.222.0.121 () {56 vars in 1122 bytes} [Thu Aug 4 10:19:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:46.539922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99b926-3ddb-4565-bf42-ba4dbfc71892 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:46.575873 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:46.622730 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfc5b9-7aab-4638-a0f3-d254b4fd4704 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:46.667742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99b926-3ddb-4565-bf42-ba4dbfc71892 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:46.680997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6883d1-5859-4f38-940a-849081e2a46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:46.681657 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6883d1-5859-4f38-940a-849081e2a46f None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:46.681973 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6883d1-5859-4f38-940a-849081e2a46f None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:46.682413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6883d1-5859-4f38-940a-849081e2a46f None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:46.683818 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6883d1-5859-4f38-940a-849081e2a46f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:46.685945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6883d1-5859-4f38-940a-849081e2a46f None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:46.704684 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 406/793] 10.222.0.121 () {56 vars in 1092 bytes} [Thu Aug 4 10:19:46 2022] GET /identity/v3/services => generated 776 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:46.706539 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6883d1-5859-4f38-940a-849081e2a46f None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:46.710405 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d0b14-a4e6-482e-a250-b991331c7652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:46.710690 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:46.712348 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d0b14-a4e6-482e-a250-b991331c7652 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:46.712859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d0b14-a4e6-482e-a250-b991331c7652 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:46.713298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d0b14-a4e6-482e-a250-b991331c7652 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:46.724348 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d0b14-a4e6-482e-a250-b991331c7652 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:46.726248 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d0b14-a4e6-482e-a250-b991331c7652 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:46.741048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-bb7d0b14-a4e6-482e-a250-b991331c7652 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 10:19:46.741048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:46.741048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:46.741048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:46.741048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:46.741048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:46.741048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:46.741048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:46.741048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:46.741048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:46.741048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:46.741048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:46.741048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:19:46.741048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:19:46.741048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:19:46.741048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:19:46.742391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:46.742391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:46.742391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:19:46.742391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:46.742391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:19:46.742391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:46.742391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:46.742391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:46.742391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:46.742391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:46.742391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:46.742391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:46.742391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:46.742391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:46.742391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:46.742391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:46.742391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:46.742391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:46.743412 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:46.743412 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:46.743412 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:19:46.743412 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:19:46.743412 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:19:46.743412 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:19:46.743412 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:19:46.743412 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:19:46.743412 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 10:19:46.743412 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:46.746091 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 388/794] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:19:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:46.757172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d0b14-a4e6-482e-a250-b991331c7652 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:46.894145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6883d1-5859-4f38-940a-849081e2a46f None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:46.927066 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:46.938716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae47c46-3b59-44b1-8075-0ce607f12816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:46.939275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae47c46-3b59-44b1-8075-0ce607f12816 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:46.939560 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae47c46-3b59-44b1-8075-0ce607f12816 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:46.939832 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae47c46-3b59-44b1-8075-0ce607f12816 None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:46.940969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae47c46-3b59-44b1-8075-0ce607f12816 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:46.941277 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae47c46-3b59-44b1-8075-0ce607f12816 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:46.942897 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae47c46-3b59-44b1-8075-0ce607f12816 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:46.966252 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 407/795] 10.222.0.121 () {56 vars in 1154 bytes} [Thu Aug 4 10:19:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:46.972210 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae47c46-3b59-44b1-8075-0ce607f12816 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:46.990592 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d0b14-a4e6-482e-a250-b991331c7652 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:47.043553 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b771e-b6b0-4ed1-b626-29a996015ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:47.044826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b771e-b6b0-4ed1-b626-29a996015ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:47.044959 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b771e-b6b0-4ed1-b626-29a996015ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:47.045191 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b771e-b6b0-4ed1-b626-29a996015ac0 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:47.046157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b771e-b6b0-4ed1-b626-29a996015ac0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:47.046482 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b771e-b6b0-4ed1-b626-29a996015ac0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:47.050768 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b771e-b6b0-4ed1-b626-29a996015ac0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:47.068305 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:47.080256 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 389/796] 10.222.0.121 () {56 vars in 1140 bytes} [Thu Aug 4 10:19:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:47.088184 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b771e-b6b0-4ed1-b626-29a996015ac0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:47.120971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae47c46-3b59-44b1-8075-0ce607f12816 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:47.177290 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef46eb-cf64-4acf-bf85-6ea82af0cf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:47.178000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef46eb-cf64-4acf-bf85-6ea82af0cf88 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:47.178280 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef46eb-cf64-4acf-bf85-6ea82af0cf88 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:47.178590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef46eb-cf64-4acf-bf85-6ea82af0cf88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:47.190430 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef46eb-cf64-4acf-bf85-6ea82af0cf88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:47.192053 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef46eb-cf64-4acf-bf85-6ea82af0cf88 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:47.208847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-29ef46eb-cf64-4acf-bf85-6ea82af0cf88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:47.208847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:47.208847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:47.208847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:47.208847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:47.208847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:47.208847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:47.208847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:47.208847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:47.208847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:47.208847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:47.208847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:47.208847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:47.208847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:47.208847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:47.210083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:47.210083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:47.210083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:47.210083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:47.210083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:47.210083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:47.210083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:47.210083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:47.210083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:47.210083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:47.210083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:47.210083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:47.210083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:47.210083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:47.210083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:47.210083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:47.210083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:47.211196 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:47.211196 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:47.211196 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:47.211196 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:47.211196 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 408/797] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:19:47.216165 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f26b59-1567-4bf7-9d47-eee19292d226 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:47.216547 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f26b59-1567-4bf7-9d47-eee19292d226 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:47.216722 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f26b59-1567-4bf7-9d47-eee19292d226 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:47.218519 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 409/798] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:47 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:47.219328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f26b59-1567-4bf7-9d47-eee19292d226 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:47.248247 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b771e-b6b0-4ed1-b626-29a996015ac0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:47.298620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe90702-828c-4637-9a44-0df118cb1106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:47.299664 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe90702-828c-4637-9a44-0df118cb1106 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:47.300174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe90702-828c-4637-9a44-0df118cb1106 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:47.300969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe90702-828c-4637-9a44-0df118cb1106 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:47.316102 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe90702-828c-4637-9a44-0df118cb1106 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:47.319170 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe90702-828c-4637-9a44-0df118cb1106 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:47.322343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:47.340701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-efe90702-828c-4637-9a44-0df118cb1106 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:47.340701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:47.340701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:47.340701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:47.340701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:47.340701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:47.340701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:47.340701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:47.340701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:47.340701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:47.340701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:47.340701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:47.340701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:47.340701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:47.340701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:47.341961 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:47.341961 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:47.341961 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:47.341961 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:47.341961 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:47.341961 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:47.341961 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:47.341961 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:47.341961 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:47.341961 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:47.341961 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:47.341961 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:47.341961 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:47.341961 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:47.341961 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:47.341961 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:47.341961 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:47.343087 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:47.343087 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:47.343087 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:47.343087 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:47.344481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 390/799] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:47.345964 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213988a6-d66e-4a32-85fc-672cad628b26 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:47.346359 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213988a6-d66e-4a32-85fc-672cad628b26 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:47.346742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213988a6-d66e-4a32-85fc-672cad628b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:47.351787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f26b59-1567-4bf7-9d47-eee19292d226 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:47.370354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:47.373384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-213988a6-d66e-4a32-85fc-672cad628b26 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:47.375725 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-213988a6-d66e-4a32-85fc-672cad628b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:47.396597 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cce530-8deb-46c4-b085-1bfda6c4505d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:47.397254 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cce530-8deb-46c4-b085-1bfda6c4505d None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:47.397545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cce530-8deb-46c4-b085-1bfda6c4505d None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:47.397828 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cce530-8deb-46c4-b085-1bfda6c4505d None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:47.398564 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cce530-8deb-46c4-b085-1bfda6c4505d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:47.398875 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cce530-8deb-46c4-b085-1bfda6c4505d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:47.400336 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cce530-8deb-46c4-b085-1bfda6c4505d None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:47.420173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 410/800] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:47.420173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cce530-8deb-46c4-b085-1bfda6c4505d None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:47.476753 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 391/801] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:47 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:19:47.478500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a70027-5cd7-49c0-a737-75325d85b4a7 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:47.478787 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a70027-5cd7-49c0-a737-75325d85b4a7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:47.479056 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a70027-5cd7-49c0-a737-75325d85b4a7 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:47.480511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 392/802] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:47.484402 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a70027-5cd7-49c0-a737-75325d85b4a7 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:47.554992 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cce530-8deb-46c4-b085-1bfda6c4505d None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:47.608426 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb872229-3f31-4719-9107-91ae6d3c7dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:47.609051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb872229-3f31-4719-9107-91ae6d3c7dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:47.610470 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb872229-3f31-4719-9107-91ae6d3c7dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:47.610470 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb872229-3f31-4719-9107-91ae6d3c7dc2 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:47.611160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb872229-3f31-4719-9107-91ae6d3c7dc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:47.611461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb872229-3f31-4719-9107-91ae6d3c7dc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:47.613854 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb872229-3f31-4719-9107-91ae6d3c7dc2 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:47.624275 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a70027-5cd7-49c0-a737-75325d85b4a7 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:47.633737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 411/803] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:47.635173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554feef-afab-441a-9db3-cdfe272b197c None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:47.635473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554feef-afab-441a-9db3-cdfe272b197c None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:47.635771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554feef-afab-441a-9db3-cdfe272b197c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:47.661476 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:47.669136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-4554feef-afab-441a-9db3-cdfe272b197c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:47.671018 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-4554feef-afab-441a-9db3-cdfe272b197c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:47.681102 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9db53b-b91d-47b5-b261-d08e9e471a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:47.681772 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9db53b-b91d-47b5-b261-d08e9e471a3d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:47.682059 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9db53b-b91d-47b5-b261-d08e9e471a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:47.682393 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9db53b-b91d-47b5-b261-d08e9e471a3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:47.691334 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9db53b-b91d-47b5-b261-d08e9e471a3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:47.692977 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9db53b-b91d-47b5-b261-d08e9e471a3d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:47.706020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-5c9db53b-b91d-47b5-b261-d08e9e471a3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:47.706020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:47.706020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:47.706020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:47.706020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:47.706020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:47.706020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:47.706020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:47.706020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:47.706020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:47.706020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:47.706020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:47.706020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:19:47.706020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:19:47.706020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:19:47.707098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:19:47.707098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:47.707098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:47.707098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:47.707098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:47.707098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:47.707098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:47.707098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:47.707098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:47.707098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:47.707098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:47.707098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:47.707098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:47.707098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:47.707098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:47.707098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:47.707098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:47.713228 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:19:47.713228 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:19:47.713228 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:19:47.713228 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:19:47.713228 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:19:47.713228 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:19:47.713228 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:47.713228 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:47.713228 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 393/804] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:47.713228 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9db53b-b91d-47b5-b261-d08e9e471a3d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:47.726926 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:47.786049 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 412/805] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:47 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:19:47.786614 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554feef-afab-441a-9db3-cdfe272b197c None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:47.855988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9db53b-b91d-47b5-b261-d08e9e471a3d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:47.899005 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94fe5b2-518a-4002-b914-9d0597d9f34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:47.899795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94fe5b2-518a-4002-b914-9d0597d9f34d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:47.900143 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94fe5b2-518a-4002-b914-9d0597d9f34d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:47.900432 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94fe5b2-518a-4002-b914-9d0597d9f34d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:47.901027 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94fe5b2-518a-4002-b914-9d0597d9f34d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:47.902434 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94fe5b2-518a-4002-b914-9d0597d9f34d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:47.912688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-f94fe5b2-518a-4002-b914-9d0597d9f34d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 10:19:47.912688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:47.912688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:19:47.912688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:19:47.912688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:19:47.912688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:19:47.912688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:19:47.912688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:19:47.912688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 10:19:47.912688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:47.912688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:19:47.912688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:47.912688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:47.912688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:47.912688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:47.912688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:47.913964 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:47.913964 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:47.913964 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:47.913964 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:47.913964 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:47.913964 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:47.913964 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:47.913964 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:19:47.913964 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:19:47.913964 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:19:47.913964 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:19:47.913964 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:47.913964 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:47.913964 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:47.913964 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:47.913964 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:47.921172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:47.921172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:47.921172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:47.921172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:47.921172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:47.921172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:47.921172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:47.921172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:47.921172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:47.921172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:47.921172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:47.921172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:19:47.921172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:19:47.921172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 10:19:47.921172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:47.922045 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554feef-afab-441a-9db3-cdfe272b197c None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:47.922142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 394/806] 10.222.0.121 () {56 vars in 1134 bytes} [Thu Aug 4 10:19:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:47.922142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94fe5b2-518a-4002-b914-9d0597d9f34d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:47.966378 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75e421-8dc8-49d9-8d7f-8f664fc691d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:47.967514 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75e421-8dc8-49d9-8d7f-8f664fc691d3 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:47.967804 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75e421-8dc8-49d9-8d7f-8f664fc691d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:47.968122 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75e421-8dc8-49d9-8d7f-8f664fc691d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:47.977975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75e421-8dc8-49d9-8d7f-8f664fc691d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:47.980580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75e421-8dc8-49d9-8d7f-8f664fc691d3 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:47.995019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-ca75e421-8dc8-49d9-8d7f-8f664fc691d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:47.995019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:47.995019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:47.995019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:47.995019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:47.995019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:47.995019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:47.995019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:47.995019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:47.995019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:47.995019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:47.995019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:47.995019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:19:47.995019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:19:47.995019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:19:47.996189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:19:47.996189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:47.996189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:47.996189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:47.996189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:47.996189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:47.996189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:47.996189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:47.996189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:47.996189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:47.996189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:47.996189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:47.996189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:47.996189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:47.996189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:47.996189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:47.996189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:47.997252 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:19:47.997252 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:19:47.997252 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:19:47.997252 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:19:47.997252 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:19:47.997252 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:19:47.997252 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:47.997252 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:48.001311 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 413/807] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:48.002264 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75e421-8dc8-49d9-8d7f-8f664fc691d3 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:48.004235 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:48.032366 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:48.067636 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94fe5b2-518a-4002-b914-9d0597d9f34d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:48.111050 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7e5a1b-fe2f-4d30-bb0b-e9844dc76a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:48.111698 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e5a1b-fe2f-4d30-bb0b-e9844dc76a0d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:48.112026 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e5a1b-fe2f-4d30-bb0b-e9844dc76a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:48.112306 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e5a1b-fe2f-4d30-bb0b-e9844dc76a0d None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:48.113449 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7e5a1b-fe2f-4d30-bb0b-e9844dc76a0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:48.113758 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7e5a1b-fe2f-4d30-bb0b-e9844dc76a0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:48.115348 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7e5a1b-fe2f-4d30-bb0b-e9844dc76a0d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:48.121558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75e421-8dc8-49d9-8d7f-8f664fc691d3 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:48.134420 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 395/808] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:19:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:48.136312 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7e5a1b-fe2f-4d30-bb0b-e9844dc76a0d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:48.163288 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12db378-e378-4dd2-9a44-6e9e3d0ddbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:48.164040 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12db378-e378-4dd2-9a44-6e9e3d0ddbb2 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:48.164277 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12db378-e378-4dd2-9a44-6e9e3d0ddbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:48.164536 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12db378-e378-4dd2-9a44-6e9e3d0ddbb2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:48.175478 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12db378-e378-4dd2-9a44-6e9e3d0ddbb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:48.181015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-b12db378-e378-4dd2-9a44-6e9e3d0ddbb2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:19:48.181015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:48.181015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:48.181015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:48.181015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:48.181015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:48.181015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:48.181015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:48.181015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:48.181015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:48.181015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:48.181015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:48.181015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:48.181015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:48.181015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:48.181015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:48.182105 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:48.182105 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:48.182105 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:48.182105 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:48.182105 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:48.182105 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:48.182105 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:48.182105 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:48.182105 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:48.182105 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:48.182105 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:48.182105 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:48.182105 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:48.182105 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:48.182105 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:48.182105 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:48.182978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:48.182978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:48.182978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:48.182978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:48.182978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:48.182978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:48.182978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:48.182978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:48.182978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:48.182978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:48.182978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:19:48.182978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:48.182978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 414/809] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:19:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:48.183698 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12db378-e378-4dd2-9a44-6e9e3d0ddbb2 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:48.265228 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7e5a1b-fe2f-4d30-bb0b-e9844dc76a0d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:48.275067 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:48.277802 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:48.295415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12db378-e378-4dd2-9a44-6e9e3d0ddbb2 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:48.310698 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec5ebe-7bd0-4736-96ee-cb213b8df642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:48.311436 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec5ebe-7bd0-4736-96ee-cb213b8df642 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:48.311590 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec5ebe-7bd0-4736-96ee-cb213b8df642 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:48.311874 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec5ebe-7bd0-4736-96ee-cb213b8df642 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:48.312643 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec5ebe-7bd0-4736-96ee-cb213b8df642 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:48.312953 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec5ebe-7bd0-4736-96ee-cb213b8df642 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:48.314578 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec5ebe-7bd0-4736-96ee-cb213b8df642 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:48.320457 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 396/810] 10.222.0.121 () {56 vars in 1129 bytes} [Thu Aug 4 10:19:48 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:48.321376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec5ebe-7bd0-4736-96ee-cb213b8df642 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:48.344504 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395850f2-519c-436a-ba28-30bb6121ee58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:48.345267 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395850f2-519c-436a-ba28-30bb6121ee58 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:48.345548 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395850f2-519c-436a-ba28-30bb6121ee58 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:48.345891 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395850f2-519c-436a-ba28-30bb6121ee58 None admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:48.346602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395850f2-519c-436a-ba28-30bb6121ee58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:48.346975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395850f2-519c-436a-ba28-30bb6121ee58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:48.349530 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395850f2-519c-436a-ba28-30bb6121ee58 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:48.363041 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 415/811] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:19:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:48.364252 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395850f2-519c-436a-ba28-30bb6121ee58 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:48.467350 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec5ebe-7bd0-4736-96ee-cb213b8df642 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:48.514557 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b994303-bad0-4e39-90df-a46cb5fdeffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:48.515327 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b994303-bad0-4e39-90df-a46cb5fdeffc None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:48.515571 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b994303-bad0-4e39-90df-a46cb5fdeffc None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:48.515812 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b994303-bad0-4e39-90df-a46cb5fdeffc None admin] PATH_INFO: `/v3/projects/22768aba760b4307aab131d1312b2bf6/users/efe6e59228bd49bfbff25a11eb682426/roles/2edb704d42834db6acd295fa3a8be5d5` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:48.534180 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b994303-bad0-4e39-90df-a46cb5fdeffc None admin] RBAC: Authorizing `identity:create_grant(project_id=22768aba760b4307aab131d1312b2bf6, user_id=efe6e59228bd49bfbff25a11eb682426, role_id=2edb704d42834db6acd295fa3a8be5d5)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:48.536949 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b994303-bad0-4e39-90df-a46cb5fdeffc None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:48.542160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395850f2-519c-436a-ba28-30bb6121ee58 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:48.554782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 397/812] 10.222.0.121 () {58 vars in 1444 bytes} [Thu Aug 4 10:19:48 2022] PUT /identity/v3/projects/22768aba760b4307aab131d1312b2bf6/users/efe6e59228bd49bfbff25a11eb682426/roles/2edb704d42834db6acd295fa3a8be5d5 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:19:48.555528 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b994303-bad0-4e39-90df-a46cb5fdeffc None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:48.611328 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:48.640995 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b1624-5fc7-4d83-8b2b-5027983ba509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:48.643864 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b1624-5fc7-4d83-8b2b-5027983ba509 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:48.644653 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b1624-5fc7-4d83-8b2b-5027983ba509 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:48.645870 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b1624-5fc7-4d83-8b2b-5027983ba509 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:48.647892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b1624-5fc7-4d83-8b2b-5027983ba509 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:48.648731 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b1624-5fc7-4d83-8b2b-5027983ba509 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:48.652265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b1624-5fc7-4d83-8b2b-5027983ba509 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:48.672485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 416/813] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:19:48 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:48.673546 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b1624-5fc7-4d83-8b2b-5027983ba509 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:48.678756 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:48.750962 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b994303-bad0-4e39-90df-a46cb5fdeffc None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:48.805183 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ab5eb-594f-44cb-a93b-de1b8d5416bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:48.805417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b1624-5fc7-4d83-8b2b-5027983ba509 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:48.805510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ab5eb-594f-44cb-a93b-de1b8d5416bf None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:48.805510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ab5eb-594f-44cb-a93b-de1b8d5416bf None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:48.805671 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ab5eb-594f-44cb-a93b-de1b8d5416bf None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:48.806792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ab5eb-594f-44cb-a93b-de1b8d5416bf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:48.806792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ab5eb-594f-44cb-a93b-de1b8d5416bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:48.808051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ab5eb-594f-44cb-a93b-de1b8d5416bf None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:48.814995 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 398/814] 10.222.0.121 () {56 vars in 1129 bytes} [Thu Aug 4 10:19:48 2022] GET /identity/v3/services?type=block-storage => generated 369 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:48.815632 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ab5eb-594f-44cb-a93b-de1b8d5416bf None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:48.855849 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4719e-acc8-4576-9e10-047692deb9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:48.857041 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4719e-acc8-4576-9e10-047692deb9eb None admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:48.857543 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4719e-acc8-4576-9e10-047692deb9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:48.858064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4719e-acc8-4576-9e10-047692deb9eb None admin] PATH_INFO: `/v3/projects/22768aba760b4307aab131d1312b2bf6/users/e16e26784601425c9883acd91f65becc/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:48.860405 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4719e-acc8-4576-9e10-047692deb9eb None admin] RBAC: Authorizing `identity:create_grant(project_id=22768aba760b4307aab131d1312b2bf6, user_id=e16e26784601425c9883acd91f65becc, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:48.862189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4719e-acc8-4576-9e10-047692deb9eb None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:48.884038 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 417/815] 10.222.0.121 () {58 vars in 1444 bytes} [Thu Aug 4 10:19:48 2022] PUT /identity/v3/projects/22768aba760b4307aab131d1312b2bf6/users/e16e26784601425c9883acd91f65becc/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:19:48.885370 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4719e-acc8-4576-9e10-047692deb9eb None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:48.961242 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:48.977621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:49.014145 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ab5eb-594f-44cb-a93b-de1b8d5416bf None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:49.030626 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4719e-acc8-4576-9e10-047692deb9eb None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:49.086772 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ebbeb-02ab-4e39-9d27-dedde07ae01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:49.087444 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ebbeb-02ab-4e39-9d27-dedde07ae01a None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:49.087734 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ebbeb-02ab-4e39-9d27-dedde07ae01a None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:49.088040 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ebbeb-02ab-4e39-9d27-dedde07ae01a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:49.088796 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ebbeb-02ab-4e39-9d27-dedde07ae01a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:49.090259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ebbeb-02ab-4e39-9d27-dedde07ae01a None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:49.115915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb8741-81c4-4473-97be-08f29baa53b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:49.116628 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb8741-81c4-4473-97be-08f29baa53b4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:49.116915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb8741-81c4-4473-97be-08f29baa53b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:49.117195 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb8741-81c4-4473-97be-08f29baa53b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:49.138095 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb8741-81c4-4473-97be-08f29baa53b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:49.140188 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb8741-81c4-4473-97be-08f29baa53b4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:49.144845 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 418/816] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:19:48 2022] POST /identity/v3/endpoints => generated 348 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:19:49.147070 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99355710-42df-4e91-94ce-5c5c7ee0e20e None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:49.147359 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99355710-42df-4e91-94ce-5c5c7ee0e20e None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:49.147637 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99355710-42df-4e91-94ce-5c5c7ee0e20e None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:49.149331 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 419/817] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:49 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:49.158332 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-25fb8741-81c4-4473-97be-08f29baa53b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:49.158332 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:49.158332 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:49.158332 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:49.158332 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:49.158332 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:49.158332 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:49.158332 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:49.158332 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:49.158332 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:49.158332 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:49.158332 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:49.158332 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:49.158332 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:49.158332 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:49.159579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:49.159579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:49.159579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:49.159579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:49.159579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:49.159579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:49.159579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:49.159579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:49.159579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:49.159579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:49.159579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:49.159579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:49.159579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:49.159579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:49.159579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:49.159579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:49.159579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:49.160795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:49.160795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:49.160795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:49.160795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:49.160795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 399/818] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:49.161097 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7e5a4-f7b4-4149-b2b7-49f08430c179 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:49.161097 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7e5a4-f7b4-4149-b2b7-49f08430c179 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:49.161097 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7e5a4-f7b4-4149-b2b7-49f08430c179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:49.170127 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb8741-81c4-4473-97be-08f29baa53b4 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:49.188418 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-f7c7e5a4-f7b4-4149-b2b7-49f08430c179 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:49.189721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-f7c7e5a4-f7b4-4149-b2b7-49f08430c179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:49.281694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 420/819] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:49 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:19:49.305461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7e5a4-f7b4-4149-b2b7-49f08430c179 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:49.360301 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb8741-81c4-4473-97be-08f29baa53b4 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:49.379665 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:49.404727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176fc1b0-bd0c-4430-9aeb-a057b8b7fc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:49.405873 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176fc1b0-bd0c-4430-9aeb-a057b8b7fc07 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:49.406429 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176fc1b0-bd0c-4430-9aeb-a057b8b7fc07 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:49.406928 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176fc1b0-bd0c-4430-9aeb-a057b8b7fc07 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:49.407985 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176fc1b0-bd0c-4430-9aeb-a057b8b7fc07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:49.408617 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176fc1b0-bd0c-4430-9aeb-a057b8b7fc07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:49.410294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176fc1b0-bd0c-4430-9aeb-a057b8b7fc07 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:49.418819 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:49.426886 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 400/820] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:49.436079 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176fc1b0-bd0c-4430-9aeb-a057b8b7fc07 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:49.446381 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7e5a4-f7b4-4149-b2b7-49f08430c179 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:49.486565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba7fa1-5215-4a4e-b145-d0416f3a1001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:49.488018 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba7fa1-5215-4a4e-b145-d0416f3a1001 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:49.488478 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba7fa1-5215-4a4e-b145-d0416f3a1001 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:49.488907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba7fa1-5215-4a4e-b145-d0416f3a1001 None admin] PATH_INFO: `/v3/services/network` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:49.489734 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba7fa1-5215-4a4e-b145-d0416f3a1001 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:49.491206 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba7fa1-5215-4a4e-b145-d0416f3a1001 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:49.500914 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-25ba7fa1-5215-4a4e-b145-d0416f3a1001 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 10:19:49.500914 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:49.500914 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:19:49.500914 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:19:49.500914 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:19:49.500914 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:19:49.500914 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:19:49.500914 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:19:49.500914 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 10:19:49.500914 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:49.500914 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:19:49.500914 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:49.500914 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:49.500914 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:49.500914 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:49.500914 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:49.500914 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:49.502332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:49.502332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:49.502332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:49.502332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:49.502332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:49.502332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:49.502332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:19:49.502332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:19:49.502332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:19:49.502332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:19:49.502332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:49.502332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:49.502332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:49.502332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:49.502332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:49.502332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:49.503417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:49.503417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:49.503417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:49.503417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:49.503417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:49.503417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:49.503417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:49.503417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:49.503417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:49.503417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:49.503417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:19:49.503417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:19:49.503417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 10:19:49.503417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:49.503417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 421/821] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:49 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:19:49.511160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba7fa1-5215-4a4e-b145-d0416f3a1001 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:49.580770 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176fc1b0-bd0c-4430-9aeb-a057b8b7fc07 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:49.625578 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ccc84-c154-4382-8ecd-e6926a1e7ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:49.626747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ccc84-c154-4382-8ecd-e6926a1e7ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:49.627267 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ccc84-c154-4382-8ecd-e6926a1e7ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:49.627752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ccc84-c154-4382-8ecd-e6926a1e7ba5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:49.636666 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ccc84-c154-4382-8ecd-e6926a1e7ba5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:49.638571 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ccc84-c154-4382-8ecd-e6926a1e7ba5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:49.650972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba7fa1-5215-4a4e-b145-d0416f3a1001 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:49.652795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-306ccc84-c154-4382-8ecd-e6926a1e7ba5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:19:49.652795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:49.652795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:49.652795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:49.652795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:49.652795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:49.652795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:49.652795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:49.652795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:49.652795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:49.652795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:49.652795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:49.652795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:19:49.652795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:19:49.652795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:19:49.652795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:19:49.653763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:49.653763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:49.653763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:19:49.653763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:49.653763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:19:49.653763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:49.653763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:49.653763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:49.653763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:49.653763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:49.653763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:49.653763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:49.653763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:49.653763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:49.653763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:49.653763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:49.653763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:49.653763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:49.654792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:49.654792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:49.654792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:19:49.654792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:19:49.654792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:19:49.654792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:19:49.654792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:19:49.654792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:19:49.654792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:19:49.654792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:49.657856 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 401/822] 10.222.0.121 () {56 vars in 1098 bytes} [Thu Aug 4 10:19:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:49.664187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ccc84-c154-4382-8ecd-e6926a1e7ba5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:49.673496 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:49.730239 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:49.734428 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8b00e-6759-42e4-af64-4f9d9ce4162c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:49.735318 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8b00e-6759-42e4-af64-4f9d9ce4162c None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:49.735437 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8b00e-6759-42e4-af64-4f9d9ce4162c None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:49.735640 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8b00e-6759-42e4-af64-4f9d9ce4162c None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:49.736472 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8b00e-6759-42e4-af64-4f9d9ce4162c None admin] RBAC: Adding query filter params (name=network) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:49.736690 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8b00e-6759-42e4-af64-4f9d9ce4162c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:49.738487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8b00e-6759-42e4-af64-4f9d9ce4162c None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:49.752546 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 422/823] 10.222.0.121 () {56 vars in 1117 bytes} [Thu Aug 4 10:19:49 2022] GET /identity/v3/services?name=network => generated 125 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:19:49.758316 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8b00e-6759-42e4-af64-4f9d9ce4162c None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:49.778810 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ccc84-c154-4382-8ecd-e6926a1e7ba5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:49.815300 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ed684-de91-4850-be24-4b12c91bd34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:49.816461 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ed684-de91-4850-be24-4b12c91bd34d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:49.816974 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ed684-de91-4850-be24-4b12c91bd34d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:49.817539 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ed684-de91-4850-be24-4b12c91bd34d None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:49.818708 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ed684-de91-4850-be24-4b12c91bd34d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:49.819228 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ed684-de91-4850-be24-4b12c91bd34d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:49.820987 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ed684-de91-4850-be24-4b12c91bd34d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:49.844513 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 402/824] 10.222.0.121 () {56 vars in 1138 bytes} [Thu Aug 4 10:19:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:19:49.850856 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ed684-de91-4850-be24-4b12c91bd34d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:49.952054 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8b00e-6759-42e4-af64-4f9d9ce4162c None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:49.988609 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb8675-a9d0-4b91-b743-afc2c4f5b878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:49.989814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb8675-a9d0-4b91-b743-afc2c4f5b878 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:49.990199 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb8675-a9d0-4b91-b743-afc2c4f5b878 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:49.990458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb8675-a9d0-4b91-b743-afc2c4f5b878 None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:49.991157 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb8675-a9d0-4b91-b743-afc2c4f5b878 None admin] RBAC: Adding query filter params (type=network) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:49.991468 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb8675-a9d0-4b91-b743-afc2c4f5b878 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:49.992927 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb8675-a9d0-4b91-b743-afc2c4f5b878 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:50.004192 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 423/825] 10.222.0.121 () {56 vars in 1117 bytes} [Thu Aug 4 10:19:49 2022] GET /identity/v3/services?type=network => generated 352 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:19:50.007142 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb8675-a9d0-4b91-b743-afc2c4f5b878 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:50.016748 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ed684-de91-4850-be24-4b12c91bd34d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:50.047199 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:50.059835 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:50.065187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f614ca7-5db9-4823-a141-c215e7cf73c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:50.066531 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f614ca7-5db9-4823-a141-c215e7cf73c0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:50.067050 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f614ca7-5db9-4823-a141-c215e7cf73c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:50.067759 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f614ca7-5db9-4823-a141-c215e7cf73c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:50.142220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb8675-a9d0-4b91-b743-afc2c4f5b878 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:50.151693 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f614ca7-5db9-4823-a141-c215e7cf73c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:50.153695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f614ca7-5db9-4823-a141-c215e7cf73c0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:50.256834 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ba58a-7334-4512-ae8c-ef57ebade292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:50.257403 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ba58a-7334-4512-ae8c-ef57ebade292 None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:50.257687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ba58a-7334-4512-ae8c-ef57ebade292 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:50.257972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ba58a-7334-4512-ae8c-ef57ebade292 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:50.258735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ba58a-7334-4512-ae8c-ef57ebade292 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:50.260178 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ba58a-7334-4512-ae8c-ef57ebade292 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:50.264249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 403/826] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:19:49 2022] GET /identity/v3/auth/tokens => generated 1003 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:19:50.270460 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f614ca7-5db9-4823-a141-c215e7cf73c0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:50.308372 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:50.308779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 424/827] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:19:50 2022] POST /identity/v3/endpoints => generated 339 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:19:50.430783 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f614ca7-5db9-4823-a141-c215e7cf73c0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:50.494379 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce226df5-d738-4eb2-94c8-368421ff73af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:50.495243 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce226df5-d738-4eb2-94c8-368421ff73af None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:50.495637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce226df5-d738-4eb2-94c8-368421ff73af None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:50.496554 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce226df5-d738-4eb2-94c8-368421ff73af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:50.509438 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce226df5-d738-4eb2-94c8-368421ff73af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:50.511011 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce226df5-d738-4eb2-94c8-368421ff73af None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:50.529148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-ce226df5-d738-4eb2-94c8-368421ff73af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:50.529148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:50.529148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:50.529148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:50.529148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:50.529148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:50.529148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:50.529148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:50.529148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:50.529148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:50.529148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:50.529148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:50.529148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:50.529148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:50.529148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:50.530384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:50.530384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:50.530384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:50.530384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:50.530384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:50.530384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:50.530384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:50.530384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:50.530384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:50.530384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:50.530384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:50.530384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:50.530384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:50.530384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:50.530384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:50.530384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:50.530384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:50.531548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:50.531548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:50.531548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:50.531548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:50.533224 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 404/828] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:19:50.539177 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ba58a-7334-4512-ae8c-ef57ebade292 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:50.705068 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:50.718208 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ba58a-7334-4512-ae8c-ef57ebade292 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:50.760549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02112f70-f9df-4800-857a-396af3d5e2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:50.761390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02112f70-f9df-4800-857a-396af3d5e2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:50.761778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02112f70-f9df-4800-857a-396af3d5e2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:50.762166 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02112f70-f9df-4800-857a-396af3d5e2f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:50.763003 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02112f70-f9df-4800-857a-396af3d5e2f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:50.763422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02112f70-f9df-4800-857a-396af3d5e2f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:50.765090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02112f70-f9df-4800-857a-396af3d5e2f9 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:50.780176 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 425/829] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:50.786217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce226df5-d738-4eb2-94c8-368421ff73af None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:50.872773 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:50.928475 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce226df5-d738-4eb2-94c8-368421ff73af None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:50.973370 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e317060-2ec3-4b66-8ee9-119f181580c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:50.974042 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e317060-2ec3-4b66-8ee9-119f181580c8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:50.974329 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e317060-2ec3-4b66-8ee9-119f181580c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:50.974672 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e317060-2ec3-4b66-8ee9-119f181580c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:50.982437 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e317060-2ec3-4b66-8ee9-119f181580c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:50.983925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e317060-2ec3-4b66-8ee9-119f181580c8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:50.996686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-9e317060-2ec3-4b66-8ee9-119f181580c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:50.996686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:50.996686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:50.996686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:50.996686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:50.996686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:50.996686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:50.996686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:50.996686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:50.996686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:50.996686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:50.996686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:50.996686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:19:50.996686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:19:50.996686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:19:51.000038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:19:51.000038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:51.000038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:51.000038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:51.000038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:51.000038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:51.000038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:51.000038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:51.000038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:51.000038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:51.000038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:51.000038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:51.000038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:51.000038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:51.000038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:51.000038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:51.000038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:51.003434 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:19:51.003434 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:19:51.003434 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:19:51.003434 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:19:51.003434 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:19:51.003434 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:19:51.003434 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:51.003434 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:51.003434 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 405/830] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:51.005563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02112f70-f9df-4800-857a-396af3d5e2f9 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:51.120270 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02112f70-f9df-4800-857a-396af3d5e2f9 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:51.158548 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11961dd3-80f5-4cf6-bb42-2ea57bdd3b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:51.159196 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11961dd3-80f5-4cf6-bb42-2ea57bdd3b26 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:51.159479 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11961dd3-80f5-4cf6-bb42-2ea57bdd3b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:51.159759 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11961dd3-80f5-4cf6-bb42-2ea57bdd3b26 None admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:51.160593 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11961dd3-80f5-4cf6-bb42-2ea57bdd3b26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:51.160905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11961dd3-80f5-4cf6-bb42-2ea57bdd3b26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:51.162660 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11961dd3-80f5-4cf6-bb42-2ea57bdd3b26 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:51.180121 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 426/831] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:19:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:51.182542 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e317060-2ec3-4b66-8ee9-119f181580c8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:51.325879 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbd0c1-c65f-4f98-9628-f70a933cff00 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:51.326428 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbd0c1-c65f-4f98-9628-f70a933cff00 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:51.326636 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbd0c1-c65f-4f98-9628-f70a933cff00 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:51.328791 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 427/832] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:51 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:51.335429 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4a67c-ac39-495e-8ac2-48fec7116e5c None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:51.335866 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4a67c-ac39-495e-8ac2-48fec7116e5c None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:51.336136 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e317060-2ec3-4b66-8ee9-119f181580c8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:51.336320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4a67c-ac39-495e-8ac2-48fec7116e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:51.358187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-73f4a67c-ac39-495e-8ac2-48fec7116e5c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:51.359445 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-73f4a67c-ac39-495e-8ac2-48fec7116e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:51.375376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:51.382650 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:51.395608 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9466ed7-7f62-49c8-8c24-d30a1ff2e80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:51.396345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9466ed7-7f62-49c8-8c24-d30a1ff2e80d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:51.396691 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9466ed7-7f62-49c8-8c24-d30a1ff2e80d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:51.396953 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9466ed7-7f62-49c8-8c24-d30a1ff2e80d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:51.398034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9466ed7-7f62-49c8-8c24-d30a1ff2e80d None admin] RBAC: Adding query filter params (role.id=7342106a255b4bce8f150726602a8b95, scope.project.id=22768aba760b4307aab131d1312b2bf6, user.id=0d947aeb749546309a8d4b7938087ba0) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:51.398367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9466ed7-7f62-49c8-8c24-d30a1ff2e80d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:51.400221 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9466ed7-7f62-49c8-8c24-d30a1ff2e80d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:51.412403 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 406/833] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:19:51 2022] GET /identity/v3/role_assignments?user.id=0d947aeb749546309a8d4b7938087ba0&scope.project.id=22768aba760b4307aab131d1312b2bf6&role.id=7342106a255b4bce8f150726602a8b95 => generated 610 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:51.413793 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb4c0d-8d0b-4c45-ae5f-777e36e31836 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:51.414101 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb4c0d-8d0b-4c45-ae5f-777e36e31836 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:51.414562 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb4c0d-8d0b-4c45-ae5f-777e36e31836 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:51.417056 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 407/834] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:51.426016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23870155-d561-4456-bdc2-be1267b5e68f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:51.426435 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23870155-d561-4456-bdc2-be1267b5e68f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:51.426605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23870155-d561-4456-bdc2-be1267b5e68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:51.449169 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-23870155-d561-4456-bdc2-be1267b5e68f None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:51.450635 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-23870155-d561-4456-bdc2-be1267b5e68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:51.469042 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 428/835] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:51 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:19:51.494093 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4a67c-ac39-495e-8ac2-48fec7116e5c None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:51.552166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 408/836] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:51 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:19:51.574721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23870155-d561-4456-bdc2-be1267b5e68f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:51.632429 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4a67c-ac39-495e-8ac2-48fec7116e5c None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:51.678549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a1c9e-6bc0-4447-a980-91b9c4c766f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:51.679252 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a1c9e-6bc0-4447-a980-91b9c4c766f1 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:51.679539 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a1c9e-6bc0-4447-a980-91b9c4c766f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:51.679822 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a1c9e-6bc0-4447-a980-91b9c4c766f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:51.695055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a1c9e-6bc0-4447-a980-91b9c4c766f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:51.700733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-752a1c9e-6bc0-4447-a980-91b9c4c766f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:51.700733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:51.700733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:51.700733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:51.700733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:51.700733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:51.700733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:51.700733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:51.700733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:51.700733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:51.700733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:51.700733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:51.700733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:51.700733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:51.700733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:51.700733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:51.701847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:51.701847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:51.701847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:51.701847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:51.701847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:51.701847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:51.701847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:51.701847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:51.701847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:51.701847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:51.701847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:51.701847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:51.701847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:51.701847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:51.701847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:51.701847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:51.702723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:51.702723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:51.702723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:51.702723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:51.702723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:51.702723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:51.702723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:51.702723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:51.702723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:51.702723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:51.702723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:19:51.702723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:51.703383 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 429/837] 10.222.0.121 () {56 vars in 1107 bytes} [Thu Aug 4 10:19:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:19:51.711679 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a1c9e-6bc0-4447-a980-91b9c4c766f1 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:51.719985 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:51.725644 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23870155-d561-4456-bdc2-be1267b5e68f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:51.755413 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:51.777400 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a78a2-4aa2-418f-ab05-23daa8db63ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:51.778091 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a78a2-4aa2-418f-ab05-23daa8db63ae None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:51.778377 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a78a2-4aa2-418f-ab05-23daa8db63ae None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:51.778669 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a78a2-4aa2-418f-ab05-23daa8db63ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:51.791902 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a78a2-4aa2-418f-ab05-23daa8db63ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:51.797071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-d67a78a2-4aa2-418f-ab05-23daa8db63ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:19:51.797071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:51.797071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:51.797071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:51.797071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:51.797071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:51.797071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:51.797071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:51.797071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:51.797071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:51.797071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:51.797071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:51.797071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:51.797071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:51.797071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:51.797071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:51.798157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:51.798157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:51.798157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:51.798157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:51.798157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:51.798157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:51.798157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:51.798157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:51.798157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:51.798157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:51.798157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:51.798157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:51.798157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:51.798157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:51.798157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:51.798157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:51.799034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:51.799034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:51.799034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:51.799034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:51.799034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:51.799034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:51.799034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:51.799034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:51.799034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:51.799034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:51.799034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:19:51.799034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:51.799034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 409/838] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:19:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:19:51.800360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb43da8-9d89-4473-baa7-32209ad4cb25 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:51.800360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb43da8-9d89-4473-baa7-32209ad4cb25 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:51.800360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb43da8-9d89-4473-baa7-32209ad4cb25 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:51.806136 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 410/839] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:51.806136 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb43da8-9d89-4473-baa7-32209ad4cb25 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:51.854123 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a1c9e-6bc0-4447-a980-91b9c4c766f1 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:51.891392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea4383-a246-4d90-bc1e-6d4520f61dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:51.892345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea4383-a246-4d90-bc1e-6d4520f61dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:51.892793 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea4383-a246-4d90-bc1e-6d4520f61dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:51.893064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea4383-a246-4d90-bc1e-6d4520f61dc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:51.893801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea4383-a246-4d90-bc1e-6d4520f61dc7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:51.894110 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea4383-a246-4d90-bc1e-6d4520f61dc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:51.895535 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea4383-a246-4d90-bc1e-6d4520f61dc7 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:51.905770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 430/840] 10.222.0.121 () {56 vars in 1108 bytes} [Thu Aug 4 10:19:51 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:51.907108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d1aa4-2577-4419-a59c-6ef0979cde86 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:51.907392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d1aa4-2577-4419-a59c-6ef0979cde86 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:51.908728 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d1aa4-2577-4419-a59c-6ef0979cde86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:51.927501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-bb5d1aa4-2577-4419-a59c-6ef0979cde86 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:51.928805 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-bb5d1aa4-2577-4419-a59c-6ef0979cde86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:51.937622 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb43da8-9d89-4473-baa7-32209ad4cb25 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:51.975378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae0c05-08f3-4e74-97ba-d963688f9da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:51.976227 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae0c05-08f3-4e74-97ba-d963688f9da8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:51.976620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae0c05-08f3-4e74-97ba-d963688f9da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:51.977009 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae0c05-08f3-4e74-97ba-d963688f9da8 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:51.977861 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae0c05-08f3-4e74-97ba-d963688f9da8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:51.978276 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae0c05-08f3-4e74-97ba-d963688f9da8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:51.979782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae0c05-08f3-4e74-97ba-d963688f9da8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:51.994758 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 411/841] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:19:51 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:19:51.994758 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae0c05-08f3-4e74-97ba-d963688f9da8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:52.003682 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:52.015189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 431/842] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:51 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:19:52.016739 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d1aa4-2577-4419-a59c-6ef0979cde86 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:52.043742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:52.110244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae0c05-08f3-4e74-97ba-d963688f9da8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:52.135449 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d1aa4-2577-4419-a59c-6ef0979cde86 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:52.148143 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0796c46-1701-42bf-ad06-5506942e48a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:52.148705 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0796c46-1701-42bf-ad06-5506942e48a5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:52.148985 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0796c46-1701-42bf-ad06-5506942e48a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:52.149264 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0796c46-1701-42bf-ad06-5506942e48a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:52.160826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0796c46-1701-42bf-ad06-5506942e48a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:52.162661 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0796c46-1701-42bf-ad06-5506942e48a5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:52.175834 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d5fd0-776e-45c1-a86e-224e749240c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:52.176632 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d5fd0-776e-45c1-a86e-224e749240c1 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:52.176915 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d5fd0-776e-45c1-a86e-224e749240c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:52.177201 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d5fd0-776e-45c1-a86e-224e749240c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:52.184815 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-b0796c46-1701-42bf-ad06-5506942e48a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:52.184815 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:52.184815 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:52.184815 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:52.184815 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:52.184815 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:52.184815 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:52.184815 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:52.184815 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:52.184815 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:52.184815 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:52.184815 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:52.184815 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:52.184815 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:52.184815 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:52.186174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:52.186174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:52.186174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:52.186174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:52.186174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:52.186174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:52.186174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:52.186174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:52.186174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:52.186174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:52.186174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:52.186174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:52.186174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:52.186174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:52.186174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:52.186174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:52.186174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:52.187103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:52.187103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:52.187103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:52.187103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:52.187103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 412/843] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:52.187617 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0796c46-1701-42bf-ad06-5506942e48a5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:52.193643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d5fd0-776e-45c1-a86e-224e749240c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:52.195144 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d5fd0-776e-45c1-a86e-224e749240c1 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:52.212770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-6c1d5fd0-776e-45c1-a86e-224e749240c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:52.212770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:52.212770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:52.212770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:52.212770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:52.212770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:52.212770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:52.212770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:52.212770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:52.212770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:52.212770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:52.212770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:52.212770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:52.212770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:52.212770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:52.213801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:52.213801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:52.213801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:52.213801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:52.213801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:52.213801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:52.213801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:52.213801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:52.213801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:52.213801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:52.213801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:52.213801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:52.213801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:52.213801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:52.213801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:52.213801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:52.213801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:52.220598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:52.220598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:52.220598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:52.220598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:52.220598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 432/844] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:52.220598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d5fd0-776e-45c1-a86e-224e749240c1 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:52.275166 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:52.293096 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:52.304608 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0796c46-1701-42bf-ad06-5506942e48a5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:52.333315 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d5fd0-776e-45c1-a86e-224e749240c1 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:52.342929 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5049b-12ea-4e8e-b87f-62607808af78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:52.344290 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5049b-12ea-4e8e-b87f-62607808af78 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:52.344803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5049b-12ea-4e8e-b87f-62607808af78 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:52.345332 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5049b-12ea-4e8e-b87f-62607808af78 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:52.346186 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5049b-12ea-4e8e-b87f-62607808af78 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:52.347913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5049b-12ea-4e8e-b87f-62607808af78 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:52.359187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-b4c5049b-12ea-4e8e-b87f-62607808af78 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 10:19:52.359187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:52.359187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:19:52.359187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:19:52.359187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:19:52.359187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:19:52.359187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:19:52.359187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:19:52.359187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 10:19:52.359187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:52.359187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:19:52.359187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:52.359187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:52.359187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:52.359187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:52.359187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:52.359187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:52.360584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:52.360584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:52.360584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:52.360584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:52.360584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:52.360584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:52.360584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:19:52.360584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:19:52.360584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:19:52.360584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:19:52.360584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:52.360584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:52.360584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:52.360584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:52.360584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:52.360584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:52.361638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:52.361638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:52.361638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:52.361638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:52.361638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:52.361638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:52.361638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:52.361638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:52.361638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:52.361638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:52.361638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:19:52.361638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:19:52.361638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 10:19:52.361638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:52.364297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 413/845] 10.222.0.121 () {56 vars in 1119 bytes} [Thu Aug 4 10:19:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:52.365339 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5049b-12ea-4e8e-b87f-62607808af78 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:52.375736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcaef04-cf38-42f9-b589-cd6ca43df5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:52.376408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcaef04-cf38-42f9-b589-cd6ca43df5ab None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:52.376693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcaef04-cf38-42f9-b589-cd6ca43df5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:52.376977 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcaef04-cf38-42f9-b589-cd6ca43df5ab None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:52.377696 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcaef04-cf38-42f9-b589-cd6ca43df5ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:52.378002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcaef04-cf38-42f9-b589-cd6ca43df5ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:52.379393 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcaef04-cf38-42f9-b589-cd6ca43df5ab None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:52.391929 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 433/846] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:52.392829 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcaef04-cf38-42f9-b589-cd6ca43df5ab None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:52.494595 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5049b-12ea-4e8e-b87f-62607808af78 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:52.520946 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcaef04-cf38-42f9-b589-cd6ca43df5ab None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:52.533948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:52.541766 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6010bd8-00c9-4cac-a87d-371be2fd9f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:52.542876 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6010bd8-00c9-4cac-a87d-371be2fd9f8b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:52.543806 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6010bd8-00c9-4cac-a87d-371be2fd9f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:52.544324 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6010bd8-00c9-4cac-a87d-371be2fd9f8b None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:52.545179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6010bd8-00c9-4cac-a87d-371be2fd9f8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:52.545636 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6010bd8-00c9-4cac-a87d-371be2fd9f8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:52.548298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6010bd8-00c9-4cac-a87d-371be2fd9f8b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:52.563045 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b28b16-1888-46d6-8776-acf690066603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:52.564098 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b28b16-1888-46d6-8776-acf690066603 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:52.564292 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b28b16-1888-46d6-8776-acf690066603 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:52.564611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b28b16-1888-46d6-8776-acf690066603 None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:52.565373 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b28b16-1888-46d6-8776-acf690066603 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:52.565955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b28b16-1888-46d6-8776-acf690066603 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:52.571463 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b28b16-1888-46d6-8776-acf690066603 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:52.572434 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 414/847] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:52.574110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6010bd8-00c9-4cac-a87d-371be2fd9f8b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:52.577155 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 434/848] 10.222.0.121 () {56 vars in 1119 bytes} [Thu Aug 4 10:19:52 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:52.581213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b28b16-1888-46d6-8776-acf690066603 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:52.606892 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:52.719481 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b28b16-1888-46d6-8776-acf690066603 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:52.725708 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6010bd8-00c9-4cac-a87d-371be2fd9f8b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:52.775175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3a5b2-f771-48fd-a310-80e0685de19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:52.775703 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3a5b2-f771-48fd-a310-80e0685de19c None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:52.776097 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3a5b2-f771-48fd-a310-80e0685de19c None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:52.776373 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3a5b2-f771-48fd-a310-80e0685de19c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:52.784722 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3a5b2-f771-48fd-a310-80e0685de19c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:52.785174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b5b41-02db-416f-94cc-cf6a840d77a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:52.785881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b5b41-02db-416f-94cc-cf6a840d77a1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:52.786165 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b5b41-02db-416f-94cc-cf6a840d77a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:52.786265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3a5b2-f771-48fd-a310-80e0685de19c None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:52.786436 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b5b41-02db-416f-94cc-cf6a840d77a1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:52.800263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-1ce3a5b2-f771-48fd-a310-80e0685de19c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 10:19:52.800263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:52.800263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:52.800263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:52.800263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:52.800263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:52.800263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:52.800263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:52.800263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:52.800263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:52.800263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:52.800263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:52.800263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:19:52.800263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:19:52.800263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:19:52.800263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:19:52.801413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:52.801413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:52.801413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:19:52.801413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:52.801413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:19:52.801413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:52.801413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:52.801413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:52.801413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:52.801413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:52.801413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:52.801413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:52.801413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:52.801413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:52.801413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:52.801413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:52.801413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:52.801413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:52.802411 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:52.802411 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:52.802411 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:19:52.802411 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:19:52.802411 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:19:52.802411 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:19:52.802411 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:19:52.802411 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:19:52.802411 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 10:19:52.802411 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:52.802411 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 435/849] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:19:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:52.802411 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3a5b2-f771-48fd-a310-80e0685de19c None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:52.807538 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4b5b41-02db-416f-94cc-cf6a840d77a1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:52.807538 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4b5b41-02db-416f-94cc-cf6a840d77a1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:52.824826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-0f4b5b41-02db-416f-94cc-cf6a840d77a1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 10:19:52.824826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:52.824826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:52.824826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:52.824826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:52.824826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:52.824826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:52.824826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:52.824826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:52.824826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:52.824826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:52.824826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:52.824826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:19:52.824826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:19:52.824826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:19:52.824826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:19:52.826108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:52.826108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:52.826108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:19:52.826108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:52.826108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:19:52.826108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:52.826108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:52.826108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:52.826108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:52.826108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:52.826108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:52.826108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:52.826108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:52.826108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:52.826108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:52.826108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:52.826108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:52.826108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:52.827429 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:52.827429 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:52.827429 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:19:52.827429 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:19:52.827429 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:19:52.827429 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:19:52.827429 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:19:52.827429 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:19:52.827429 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 10:19:52.827429 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:52.830794 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 415/850] 10.222.0.121 () {56 vars in 1122 bytes} [Thu Aug 4 10:19:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:52.832034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b5b41-02db-416f-94cc-cf6a840d77a1 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:52.849094 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:52.934822 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:52.935067 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3a5b2-f771-48fd-a310-80e0685de19c None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:52.963062 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b5b41-02db-416f-94cc-cf6a840d77a1 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:52.982080 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d7614e-83fd-497a-bed0-f29dc4c46c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:52.982722 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7614e-83fd-497a-bed0-f29dc4c46c8d None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:52.983026 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7614e-83fd-497a-bed0-f29dc4c46c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:52.983798 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7614e-83fd-497a-bed0-f29dc4c46c8d None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:52.984828 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d7614e-83fd-497a-bed0-f29dc4c46c8d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:52.985147 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d7614e-83fd-497a-bed0-f29dc4c46c8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:52.987031 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d7614e-83fd-497a-bed0-f29dc4c46c8d None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:53.003989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 436/851] 10.222.0.121 () {56 vars in 1119 bytes} [Thu Aug 4 10:19:52 2022] GET /identity/v3/services?type=volumev3 => generated 126 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:53.003989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d7614e-83fd-497a-bed0-f29dc4c46c8d None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:53.008540 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9aa59e-2b5f-48ad-8258-604350b9330a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:53.009298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9aa59e-2b5f-48ad-8258-604350b9330a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:53.009640 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9aa59e-2b5f-48ad-8258-604350b9330a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:53.009983 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9aa59e-2b5f-48ad-8258-604350b9330a None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:53.010884 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9aa59e-2b5f-48ad-8258-604350b9330a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:53.011261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9aa59e-2b5f-48ad-8258-604350b9330a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:53.012909 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9aa59e-2b5f-48ad-8258-604350b9330a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:53.035472 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 416/852] 10.222.0.121 () {56 vars in 1140 bytes} [Thu Aug 4 10:19:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:53.042083 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9aa59e-2b5f-48ad-8258-604350b9330a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:53.148650 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:53.157368 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d7614e-83fd-497a-bed0-f29dc4c46c8d None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:53.187885 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9aa59e-2b5f-48ad-8258-604350b9330a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:53.200155 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbdedc1-7190-48d9-a428-6ee6f3a22007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:53.200830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdedc1-7190-48d9-a428-6ee6f3a22007 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:53.201112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdedc1-7190-48d9-a428-6ee6f3a22007 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:53.201391 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdedc1-7190-48d9-a428-6ee6f3a22007 None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:53.202351 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbdedc1-7190-48d9-a428-6ee6f3a22007 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:53.202659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbdedc1-7190-48d9-a428-6ee6f3a22007 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:53.204719 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbdedc1-7190-48d9-a428-6ee6f3a22007 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:53.229508 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 437/853] 10.222.0.121 () {56 vars in 1154 bytes} [Thu Aug 4 10:19:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:53.236593 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbdedc1-7190-48d9-a428-6ee6f3a22007 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:53.238505 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272531d1-cbe4-401d-a116-31b670ddb2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:53.239204 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272531d1-cbe4-401d-a116-31b670ddb2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:53.239485 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272531d1-cbe4-401d-a116-31b670ddb2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:53.239756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272531d1-cbe4-401d-a116-31b670ddb2c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:53.242942 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:53.326044 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272531d1-cbe4-401d-a116-31b670ddb2c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:53.329469 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272531d1-cbe4-401d-a116-31b670ddb2c8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:53.376966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbdedc1-7190-48d9-a428-6ee6f3a22007 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:53.425143 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5835723-8042-4812-955a-437a842099c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:53.426134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5835723-8042-4812-955a-437a842099c9 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:53.426568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5835723-8042-4812-955a-437a842099c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:53.426993 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5835723-8042-4812-955a-437a842099c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:53.454344 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:53.455986 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 417/854] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:19:53 2022] GET /identity/v3/auth/tokens => generated 1163 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:19:53.462185 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272531d1-cbe4-401d-a116-31b670ddb2c8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:53.509274 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5835723-8042-4812-955a-437a842099c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:53.511275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5835723-8042-4812-955a-437a842099c9 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:53.512533 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:53.600128 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272531d1-cbe4-401d-a116-31b670ddb2c8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:53.642791 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646b8ad-a893-4487-a2a7-b391ed260592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:53.643570 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646b8ad-a893-4487-a2a7-b391ed260592 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:53.643955 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646b8ad-a893-4487-a2a7-b391ed260592 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:53.644323 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646b8ad-a893-4487-a2a7-b391ed260592 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:53.656549 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0646b8ad-a893-4487-a2a7-b391ed260592 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:53.658997 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0646b8ad-a893-4487-a2a7-b391ed260592 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:53.663921 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 438/855] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:19:53 2022] GET /identity/v3/auth/tokens => generated 1163 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:19:53.673625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa82bb-a166-478d-8059-36f8533b4d99 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:53.673934 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa82bb-a166-478d-8059-36f8533b4d99 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:53.674191 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa82bb-a166-478d-8059-36f8533b4d99 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:53.675609 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 439/856] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:53.680443 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa82bb-a166-478d-8059-36f8533b4d99 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:53.681158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-0646b8ad-a893-4487-a2a7-b391ed260592 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:53.681158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:53.681158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:53.681158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:53.681158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:53.681158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:53.681158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:53.681158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:53.681158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:53.681158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:53.681158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:53.681158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:53.681158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:53.681158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:53.681158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:53.682060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:53.682060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:53.682060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:53.682060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:53.682060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:53.682060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:53.682060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:53.682060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:53.682060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:53.682060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:53.682060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:53.682060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:53.682060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:53.682060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:53.682060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:53.682060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:53.682060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:53.683037 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:53.683037 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:53.683037 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:53.683037 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:53.683037 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 418/857] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:53.688316 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76eb7f1-6d31-4ae6-b6c3-d4002041199b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:53.688316 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76eb7f1-6d31-4ae6-b6c3-d4002041199b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:53.688316 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76eb7f1-6d31-4ae6-b6c3-d4002041199b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:53.708521 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d76eb7f1-6d31-4ae6-b6c3-d4002041199b None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:53.710026 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d76eb7f1-6d31-4ae6-b6c3-d4002041199b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:53.745092 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:53.790504 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 419/858] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:53 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:19:53.791147 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76eb7f1-6d31-4ae6-b6c3-d4002041199b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:53.817342 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:53.817585 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa82bb-a166-478d-8059-36f8533b4d99 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:53.859621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7604fb6-c36f-44a8-914a-acdd9268ce13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:53.861235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7604fb6-c36f-44a8-914a-acdd9268ce13 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:53.861532 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7604fb6-c36f-44a8-914a-acdd9268ce13 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:53.862102 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7604fb6-c36f-44a8-914a-acdd9268ce13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:53.873825 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7604fb6-c36f-44a8-914a-acdd9268ce13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:53.875346 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7604fb6-c36f-44a8-914a-acdd9268ce13 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:53.892916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-d7604fb6-c36f-44a8-914a-acdd9268ce13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:53.892916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:53.892916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:53.892916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:53.892916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:53.892916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:53.892916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:53.892916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:53.892916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:53.892916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:53.892916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:53.892916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:53.892916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:53.892916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:53.892916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:53.893966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:53.893966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:53.893966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:53.893966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:53.893966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:53.893966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:53.893966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:53.893966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:53.893966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:53.893966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:53.893966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:53.893966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:53.893966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:53.893966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:53.893966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:53.893966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:53.893966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:53.894911 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:53.894911 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:53.894911 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:53.894911 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:53.894911 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 440/859] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:53.895217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7604fb6-c36f-44a8-914a-acdd9268ce13 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:53.936737 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76eb7f1-6d31-4ae6-b6c3-d4002041199b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:53.983895 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37057374-6271-4c42-94da-750f44ab7908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:53.984742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37057374-6271-4c42-94da-750f44ab7908 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:53.985035 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37057374-6271-4c42-94da-750f44ab7908 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:53.985310 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37057374-6271-4c42-94da-750f44ab7908 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:53.986055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37057374-6271-4c42-94da-750f44ab7908 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:53.986450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37057374-6271-4c42-94da-750f44ab7908 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:53.988242 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37057374-6271-4c42-94da-750f44ab7908 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:54.010255 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 420/860] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:54.010255 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37057374-6271-4c42-94da-750f44ab7908 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:54.015657 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7604fb6-c36f-44a8-914a-acdd9268ce13 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:54.024788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:54.054290 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd056d6-af43-4f13-8641-57eb65af4044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:54.054958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd056d6-af43-4f13-8641-57eb65af4044 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:54.055241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd056d6-af43-4f13-8641-57eb65af4044 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:54.055514 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd056d6-af43-4f13-8641-57eb65af4044 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:54.056237 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd056d6-af43-4f13-8641-57eb65af4044 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:54.057665 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd056d6-af43-4f13-8641-57eb65af4044 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:54.068683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-0dd056d6-af43-4f13-8641-57eb65af4044 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 10:19:54.068683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:54.068683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:19:54.068683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:19:54.068683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:19:54.068683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:19:54.068683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:19:54.068683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:19:54.068683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 10:19:54.068683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:54.068683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:19:54.068683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:54.068683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:54.068683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:54.068683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:54.068683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:54.069769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:54.069769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:54.069769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:54.069769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:54.069769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:54.069769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:54.069769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:54.069769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:19:54.069769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:19:54.069769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:19:54.069769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:19:54.069769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:54.069769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:54.069769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:54.069769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:54.069769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:54.070786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:54.070786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:54.070786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:54.070786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:54.070786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:54.070786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:54.070786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:54.070786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:54.070786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:54.070786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:54.070786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:54.070786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:19:54.070786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:19:54.070786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 10:19:54.070786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:54.075115 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 441/861] 10.222.0.121 () {56 vars in 1137 bytes} [Thu Aug 4 10:19:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:54.076047 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd056d6-af43-4f13-8641-57eb65af4044 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:54.133255 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:54.161108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37057374-6271-4c42-94da-750f44ab7908 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:54.196014 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd056d6-af43-4f13-8641-57eb65af4044 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:54.207866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ebc365-59ff-4ab9-aa38-a409a2c2e26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:54.208565 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ebc365-59ff-4ab9-aa38-a409a2c2e26c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:54.209019 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ebc365-59ff-4ab9-aa38-a409a2c2e26c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:54.209306 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ebc365-59ff-4ab9-aa38-a409a2c2e26c None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:54.210368 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ebc365-59ff-4ab9-aa38-a409a2c2e26c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:54.210368 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ebc365-59ff-4ab9-aa38-a409a2c2e26c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:54.211710 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ebc365-59ff-4ab9-aa38-a409a2c2e26c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:54.232003 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 421/862] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:54.232003 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ebc365-59ff-4ab9-aa38-a409a2c2e26c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:54.236494 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba76a4-eeed-4ea8-b630-040ca118319a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:54.237216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba76a4-eeed-4ea8-b630-040ca118319a None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:54.237491 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba76a4-eeed-4ea8-b630-040ca118319a None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:54.238078 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba76a4-eeed-4ea8-b630-040ca118319a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:54.246030 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba76a4-eeed-4ea8-b630-040ca118319a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:54.247517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba76a4-eeed-4ea8-b630-040ca118319a None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:54.259923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-06ba76a4-eeed-4ea8-b630-040ca118319a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:54.259923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:54.259923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:54.259923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:54.259923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:54.259923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:54.259923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:54.259923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:54.259923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:54.259923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:54.259923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:54.259923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:54.259923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:19:54.259923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:19:54.259923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:19:54.261213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:19:54.261213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:54.261213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:54.261213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:54.261213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:54.261213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:54.261213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:54.261213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:54.261213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:54.261213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:54.261213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:54.261213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:54.261213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:54.261213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:54.261213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:54.261213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:54.261213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:54.262170 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:19:54.262170 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:19:54.262170 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:19:54.262170 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:19:54.262170 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:19:54.262170 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:19:54.262170 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:54.262170 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:54.266246 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 442/863] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:54.267305 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba76a4-eeed-4ea8-b630-040ca118319a None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:54.306661 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:54.355724 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ebc365-59ff-4ab9-aa38-a409a2c2e26c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:54.391452 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba76a4-eeed-4ea8-b630-040ca118319a None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:54.398186 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cea0f7-f841-47fd-a68c-60d103de9f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:54.398819 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cea0f7-f841-47fd-a68c-60d103de9f02 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:54.399201 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cea0f7-f841-47fd-a68c-60d103de9f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:54.399488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cea0f7-f841-47fd-a68c-60d103de9f02 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:54.400221 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cea0f7-f841-47fd-a68c-60d103de9f02 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:54.400516 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cea0f7-f841-47fd-a68c-60d103de9f02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:54.402047 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cea0f7-f841-47fd-a68c-60d103de9f02 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:54.405971 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:54.413924 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 422/864] 10.222.0.121 () {56 vars in 1131 bytes} [Thu Aug 4 10:19:54 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:54.413924 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cea0f7-f841-47fd-a68c-60d103de9f02 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:54.437136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428232d4-28a4-4d0d-9545-b34778bbdd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:54.437932 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428232d4-28a4-4d0d-9545-b34778bbdd82 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:54.438216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428232d4-28a4-4d0d-9545-b34778bbdd82 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:54.438499 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428232d4-28a4-4d0d-9545-b34778bbdd82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:54.446105 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428232d4-28a4-4d0d-9545-b34778bbdd82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:54.447857 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428232d4-28a4-4d0d-9545-b34778bbdd82 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:54.460735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-428232d4-28a4-4d0d-9545-b34778bbdd82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:54.460735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:54.460735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:54.460735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:54.460735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:54.460735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:54.460735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:54.460735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:54.460735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:54.460735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:54.460735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:54.460735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:54.460735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:19:54.460735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:19:54.460735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:19:54.461787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:19:54.461787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:54.461787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:54.461787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:54.461787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:54.461787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:54.461787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:54.461787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:54.461787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:54.461787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:54.461787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:54.461787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:54.461787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:54.461787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:54.461787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:54.461787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:54.461787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:54.462724 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:19:54.462724 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:19:54.462724 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:19:54.462724 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:19:54.462724 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:19:54.462724 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:19:54.462724 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:19:54.462724 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:54.462724 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 443/865] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:19:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:54.463247 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428232d4-28a4-4d0d-9545-b34778bbdd82 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:54.540444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cea0f7-f841-47fd-a68c-60d103de9f02 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:54.571232 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:54.586622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428232d4-28a4-4d0d-9545-b34778bbdd82 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:54.588132 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f42994-54ef-45e0-a0f6-c55e3db23765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:54.588781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f42994-54ef-45e0-a0f6-c55e3db23765 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:54.589062 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f42994-54ef-45e0-a0f6-c55e3db23765 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:54.589338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f42994-54ef-45e0-a0f6-c55e3db23765 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:54.590108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f42994-54ef-45e0-a0f6-c55e3db23765 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:54.590423 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f42994-54ef-45e0-a0f6-c55e3db23765 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:54.591894 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f42994-54ef-45e0-a0f6-c55e3db23765 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:54.605069 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 423/866] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:19:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:54.605919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f42994-54ef-45e0-a0f6-c55e3db23765 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:54.635698 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c66a1b9-fe8f-49ca-a5ba-8574751bbea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:54.636374 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66a1b9-fe8f-49ca-a5ba-8574751bbea0 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:54.636659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66a1b9-fe8f-49ca-a5ba-8574751bbea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:54.636929 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66a1b9-fe8f-49ca-a5ba-8574751bbea0 None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:54.637619 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c66a1b9-fe8f-49ca-a5ba-8574751bbea0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:54.637923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c66a1b9-fe8f-49ca-a5ba-8574751bbea0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:54.639318 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c66a1b9-fe8f-49ca-a5ba-8574751bbea0 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:54.644754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 444/867] 10.222.0.121 () {56 vars in 1131 bytes} [Thu Aug 4 10:19:54 2022] GET /identity/v3/services?type=compute_legacy => generated 132 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:54.645536 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c66a1b9-fe8f-49ca-a5ba-8574751bbea0 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:54.684476 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:54.742115 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f42994-54ef-45e0-a0f6-c55e3db23765 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:54.791471 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36389c-8b74-4ecb-b5f4-734392c0fad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:54.792781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36389c-8b74-4ecb-b5f4-734392c0fad9 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:54.793830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36389c-8b74-4ecb-b5f4-734392c0fad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:54.794622 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36389c-8b74-4ecb-b5f4-734392c0fad9 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:54.795641 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36389c-8b74-4ecb-b5f4-734392c0fad9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:54.796203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36389c-8b74-4ecb-b5f4-734392c0fad9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:54.796581 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c66a1b9-fe8f-49ca-a5ba-8574751bbea0 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:54.798904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36389c-8b74-4ecb-b5f4-734392c0fad9 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:54.822028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 424/868] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:19:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:54.828296 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36389c-8b74-4ecb-b5f4-734392c0fad9 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:54.845430 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbd844-46e3-4198-8de5-69e5a4d4add1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:54.845430 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbd844-46e3-4198-8de5-69e5a4d4add1 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:54.845795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbd844-46e3-4198-8de5-69e5a4d4add1 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:54.846012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbd844-46e3-4198-8de5-69e5a4d4add1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:54.846860 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bbd844-46e3-4198-8de5-69e5a4d4add1 None admin] RBAC: Adding query filter params (role.id=7342106a255b4bce8f150726602a8b95, scope.project.id=22768aba760b4307aab131d1312b2bf6, user.id=e16e26784601425c9883acd91f65becc) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:54.847177 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bbd844-46e3-4198-8de5-69e5a4d4add1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:54.849422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bbd844-46e3-4198-8de5-69e5a4d4add1 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:54.858330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 445/869] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:19:54 2022] GET /identity/v3/role_assignments?user.id=e16e26784601425c9883acd91f65becc&scope.project.id=22768aba760b4307aab131d1312b2bf6&role.id=7342106a255b4bce8f150726602a8b95 => generated 610 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:54.971542 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36389c-8b74-4ecb-b5f4-734392c0fad9 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:55.000632 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:55.021567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a490b-a284-429f-b5e4-127fbc5644af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:55.022219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a490b-a284-429f-b5e4-127fbc5644af None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:55.022502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a490b-a284-429f-b5e4-127fbc5644af None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:55.022778 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a490b-a284-429f-b5e4-127fbc5644af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:55.023614 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0a490b-a284-429f-b5e4-127fbc5644af None admin] RBAC: Adding query filter params (role.id=2edb704d42834db6acd295fa3a8be5d5, scope.project.id=22768aba760b4307aab131d1312b2bf6, user.id=efe6e59228bd49bfbff25a11eb682426) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:55.023929 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0a490b-a284-429f-b5e4-127fbc5644af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:55.025622 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0a490b-a284-429f-b5e4-127fbc5644af None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:55.033410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 425/870] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:19:54 2022] GET /identity/v3/role_assignments?user.id=efe6e59228bd49bfbff25a11eb682426&scope.project.id=22768aba760b4307aab131d1312b2bf6&role.id=2edb704d42834db6acd295fa3a8be5d5 => generated 610 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:55.198506 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c862a-2e86-408c-8624-bf091f00de0a None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:55.198983 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c862a-2e86-408c-8624-bf091f00de0a None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:55.199095 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c862a-2e86-408c-8624-bf091f00de0a None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:55.200805 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 446/871] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:55 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:55.209230 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a0ac4-1bad-450b-9f20-47d6adac952c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:55.209877 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a0ac4-1bad-450b-9f20-47d6adac952c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:55.210039 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a0ac4-1bad-450b-9f20-47d6adac952c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:55.230836 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-728a0ac4-1bad-450b-9f20-47d6adac952c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:55.232013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-728a0ac4-1bad-450b-9f20-47d6adac952c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:55.322668 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 426/872] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:55 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:19:55.348331 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c862a-2e86-408c-8624-bf091f00de0a None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:55.371698 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:55.483060 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c862a-2e86-408c-8624-bf091f00de0a None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:55.544827 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafbd7fa-6546-4139-82a8-0263287e73f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:55.546311 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbd7fa-6546-4139-82a8-0263287e73f4 None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:55.547049 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbd7fa-6546-4139-82a8-0263287e73f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:55.547934 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbd7fa-6546-4139-82a8-0263287e73f4 None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:55.549542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafbd7fa-6546-4139-82a8-0263287e73f4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:55.552639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafbd7fa-6546-4139-82a8-0263287e73f4 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:55.581634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 447/873] 10.222.0.121 () {60 vars in 1146 bytes} [Thu Aug 4 10:19:55 2022] POST /identity/v3/services => generated 252 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:19:56.844277 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80c022-49f8-4948-a799-b040e3243454 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:56.844845 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80c022-49f8-4948-a799-b040e3243454 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:56.844845 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80c022-49f8-4948-a799-b040e3243454 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:56.846434 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 427/874] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:56.852416 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da81c4-f752-4731-adf5-23f77c5c042f None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:56.853031 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da81c4-f752-4731-adf5-23f77c5c042f None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:56.853031 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da81c4-f752-4731-adf5-23f77c5c042f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:56.874953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-e9da81c4-f752-4731-adf5-23f77c5c042f None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:56.876478 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-e9da81c4-f752-4731-adf5-23f77c5c042f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:56.991035 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 448/875] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:56 2022] POST /identity/v3/auth/tokens => generated 1264 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:19:57.015300 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80c022-49f8-4948-a799-b040e3243454 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:57.130450 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b923211-557d-4fa1-b748-d864414bc366 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:57.131224 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b923211-557d-4fa1-b748-d864414bc366 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:57.131730 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b923211-557d-4fa1-b748-d864414bc366 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:57.133437 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 449/876] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:57 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:57.140244 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014455e-7a68-4839-bb84-224b49b9048d None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:57.140590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014455e-7a68-4839-bb84-224b49b9048d None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:57.140798 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014455e-7a68-4839-bb84-224b49b9048d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:57.149366 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:57.150475 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80c022-49f8-4948-a799-b040e3243454 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:57.163229 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-4014455e-7a68-4839-bb84-224b49b9048d None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:57.164558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-4014455e-7a68-4839-bb84-224b49b9048d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:57.168939 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:57.191711 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fca69-ac02-4601-a694-3eb6093c09d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:57.192386 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fca69-ac02-4601-a694-3eb6093c09d4 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:57.192669 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fca69-ac02-4601-a694-3eb6093c09d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:57.192979 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fca69-ac02-4601-a694-3eb6093c09d4 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:57.193737 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6fca69-ac02-4601-a694-3eb6093c09d4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:57.195150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6fca69-ac02-4601-a694-3eb6093c09d4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:57.217891 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 428/877] 10.222.0.121 () {60 vars in 1146 bytes} [Thu Aug 4 10:19:57 2022] POST /identity/v3/services => generated 270 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:19:57.300544 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 450/878] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:57 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:19:57.323175 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fca69-ac02-4601-a694-3eb6093c09d4 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:57.455015 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fca69-ac02-4601-a694-3eb6093c09d4 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:57.495584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc72490-9eae-493f-9fe7-1bc89b1a5c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:57.496289 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc72490-9eae-493f-9fe7-1bc89b1a5c4b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:57.496575 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc72490-9eae-493f-9fe7-1bc89b1a5c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:57.496843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc72490-9eae-493f-9fe7-1bc89b1a5c4b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:57.497496 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc72490-9eae-493f-9fe7-1bc89b1a5c4b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:57.499134 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc72490-9eae-493f-9fe7-1bc89b1a5c4b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:57.509502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-fbc72490-9eae-493f-9fe7-1bc89b1a5c4b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:19:57.509502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:57.509502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:19:57.509502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:19:57.509502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:19:57.509502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:19:57.509502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:19:57.509502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:19:57.509502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:19:57.509502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:57.509502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:19:57.509502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:57.509502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:57.509502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:57.509502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:57.509502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:57.509502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:57.510653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:57.510653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:57.510653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:57.510653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:57.510653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:57.510653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:57.510653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:19:57.510653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:19:57.510653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:19:57.510653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:19:57.510653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:57.510653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:57.510653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:57.510653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:57.510653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:57.510653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:57.511530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:57.511530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:57.511530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:57.511530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:57.511530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:57.511530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:57.511530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:57.511530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:57.511530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:57.511530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:57.511530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:19:57.511530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:19:57.511530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:19:57.511530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:57.511530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 429/879] 10.222.0.121 () {56 vars in 1131 bytes} [Thu Aug 4 10:19:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:57.517502 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014455e-7a68-4839-bb84-224b49b9048d None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:57.641061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014455e-7a68-4839-bb84-224b49b9048d None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:57.674405 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:57.681392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c3bdd-8954-4bca-9ba1-5316c33e1d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:57.682224 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c3bdd-8954-4bca-9ba1-5316c33e1d23 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:57.682613 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c3bdd-8954-4bca-9ba1-5316c33e1d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:57.682863 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c3bdd-8954-4bca-9ba1-5316c33e1d23 None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:57.683584 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c3bdd-8954-4bca-9ba1-5316c33e1d23 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:57.684270 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c3bdd-8954-4bca-9ba1-5316c33e1d23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:57.686199 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c3bdd-8954-4bca-9ba1-5316c33e1d23 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:57.696313 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 451/880] 10.222.0.121 () {56 vars in 1127 bytes} [Thu Aug 4 10:19:57 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:57.699404 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc72490-9eae-493f-9fe7-1bc89b1a5c4b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:57.725063 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924524c-96e8-47f0-8e12-9c71d97be7b5 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:57.725387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924524c-96e8-47f0-8e12-9c71d97be7b5 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:57.725861 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924524c-96e8-47f0-8e12-9c71d97be7b5 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:57.734865 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 452/881] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:57 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 10:19:57.737931 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e53c5-685a-4f19-aef9-ce074e61e5df None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:57.738428 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e53c5-685a-4f19-aef9-ce074e61e5df None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:57.738921 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e53c5-685a-4f19-aef9-ce074e61e5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:57.769644 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-926e53c5-685a-4f19-aef9-ce074e61e5df None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:57.771659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-926e53c5-685a-4f19-aef9-ce074e61e5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:57.777220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:57.889259 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc72490-9eae-493f-9fe7-1bc89b1a5c4b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:57.929871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81588240-57b8-4ed0-95cd-0b784ee38570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:57.930505 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81588240-57b8-4ed0-95cd-0b784ee38570 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:57.930831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81588240-57b8-4ed0-95cd-0b784ee38570 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:57.931110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81588240-57b8-4ed0-95cd-0b784ee38570 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:57.931749 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81588240-57b8-4ed0-95cd-0b784ee38570 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:57.935976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81588240-57b8-4ed0-95cd-0b784ee38570 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:57.936195 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 453/882] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:57 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:19:57.936949 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81588240-57b8-4ed0-95cd-0b784ee38570 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:57.939518 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 430/883] 10.222.0.121 () {56 vars in 1127 bytes} [Thu Aug 4 10:19:57 2022] GET /identity/v3/services?type=object-store => generated 130 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:19:57.968422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e53c5-685a-4f19-aef9-ce074e61e5df None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:58.006685 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3595e1-d9a7-44a3-8dd6-7fe63ea03a31 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:58.007677 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3595e1-d9a7-44a3-8dd6-7fe63ea03a31 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:58.008512 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3595e1-d9a7-44a3-8dd6-7fe63ea03a31 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:58.018383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 431/884] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:58 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:58.021349 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd857119-229d-409e-bce0-7e7e880f6293 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:58.022262 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd857119-229d-409e-bce0-7e7e880f6293 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:58.023023 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd857119-229d-409e-bce0-7e7e880f6293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:58.055838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-dd857119-229d-409e-bce0-7e7e880f6293 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:58.058737 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-dd857119-229d-409e-bce0-7e7e880f6293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:58.156851 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e53c5-685a-4f19-aef9-ce074e61e5df None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:58.178803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 432/885] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:58 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:19:58.189124 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:58.208922 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb27b3-c966-4a29-87a4-9485bba8bf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:58.209669 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb27b3-c966-4a29-87a4-9485bba8bf02 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:58.209961 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb27b3-c966-4a29-87a4-9485bba8bf02 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:58.210244 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb27b3-c966-4a29-87a4-9485bba8bf02 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:58.222785 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb27b3-c966-4a29-87a4-9485bba8bf02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:19:58.228265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-1eeb27b3-c966-4a29-87a4-9485bba8bf02 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 10:19:58.228265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:58.228265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:58.228265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:58.228265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:58.228265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:58.228265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:58.228265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:58.228265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:58.228265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:58.228265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:58.228265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:58.228265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:19:58.228265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:19:58.228265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:19:58.228265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:19:58.229337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:19:58.229337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:19:58.229337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:19:58.229337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:19:58.229337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:58.229337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:58.229337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:58.229337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:58.229337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:58.229337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:58.229337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:58.229337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:58.229337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:58.229337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:58.229337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:58.229337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:58.230223 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:58.230223 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:58.230223 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:58.230223 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:58.230223 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:19:58.230223 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:19:58.230223 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:19:58.230223 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:19:58.230223 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:19:58.230223 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:19:58.230223 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 10:19:58.230223 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:58.230223 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 454/886] 10.222.0.121 () {56 vars in 1125 bytes} [Thu Aug 4 10:19:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:58.234041 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd857119-229d-409e-bce0-7e7e880f6293 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:58.236086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb27b3-c966-4a29-87a4-9485bba8bf02 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:58.276645 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:58.376904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd857119-229d-409e-bce0-7e7e880f6293 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:58.384959 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb27b3-c966-4a29-87a4-9485bba8bf02 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:58.432355 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1273ab-5e9f-4ddb-bfda-17c25bb17ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:58.433464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1273ab-5e9f-4ddb-bfda-17c25bb17ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:58.433757 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1273ab-5e9f-4ddb-bfda-17c25bb17ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:58.434038 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1273ab-5e9f-4ddb-bfda-17c25bb17ec6 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:58.435085 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1273ab-5e9f-4ddb-bfda-17c25bb17ec6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:58.435401 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1273ab-5e9f-4ddb-bfda-17c25bb17ec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:58.436972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1273ab-5e9f-4ddb-bfda-17c25bb17ec6 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:58.441129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9106884-8c76-4bf3-b0f0-27d1aed22eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:58.442333 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9106884-8c76-4bf3-b0f0-27d1aed22eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:58.442824 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9106884-8c76-4bf3-b0f0-27d1aed22eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:58.443387 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9106884-8c76-4bf3-b0f0-27d1aed22eb4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:58.444630 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9106884-8c76-4bf3-b0f0-27d1aed22eb4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:58.447169 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9106884-8c76-4bf3-b0f0-27d1aed22eb4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:58.452648 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 455/887] 10.222.0.121 () {56 vars in 1120 bytes} [Thu Aug 4 10:19:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:19:58.453690 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1273ab-5e9f-4ddb-bfda-17c25bb17ec6 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:58.468070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-a9106884-8c76-4bf3-b0f0-27d1aed22eb4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 10:19:58.468070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:58.468070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:19:58.468070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:19:58.468070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:19:58.468070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:19:58.468070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:19:58.468070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:19:58.468070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 10:19:58.468070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:58.468070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:19:58.468070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:58.468070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:58.468070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:58.468070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:58.468070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:58.468070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:58.470545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:58.470545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:58.470545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:58.470545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:58.470545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:58.470545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:58.470545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:19:58.470545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:19:58.470545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:19:58.470545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:19:58.470545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:58.470545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:58.470545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:58.470545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:58.470545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:58.470545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:58.472605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:58.472605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:58.472605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:58.472605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:58.472605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:58.472605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:58.472605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:58.472605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:58.472605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:58.472605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:58.472605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:19:58.472605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:19:58.472605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 10:19:58.472605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:19:58.474023 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 433/888] 10.222.0.121 () {56 vars in 1119 bytes} [Thu Aug 4 10:19:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:58.479456 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9106884-8c76-4bf3-b0f0-27d1aed22eb4 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:58.516813 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:58.601576 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1273ab-5e9f-4ddb-bfda-17c25bb17ec6 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:58.656383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:58.660256 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17913e3b-0f27-4944-be31-c079d9956f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:58.661705 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17913e3b-0f27-4944-be31-c079d9956f9b None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:58.662765 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17913e3b-0f27-4944-be31-c079d9956f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:58.663294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17913e3b-0f27-4944-be31-c079d9956f9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:58.668933 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9106884-8c76-4bf3-b0f0-27d1aed22eb4 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:58.685214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17913e3b-0f27-4944-be31-c079d9956f9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:58.688633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17913e3b-0f27-4944-be31-c079d9956f9b None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:58.716187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-17913e3b-0f27-4944-be31-c079d9956f9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:58.716187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:58.716187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:58.716187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:58.716187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:58.716187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:58.716187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:58.716187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:58.716187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:58.716187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:58.716187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:58.716187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:58.716187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:19:58.716187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:19:58.716187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:19:58.717753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:19:58.717753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:58.717753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:58.717753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:58.717753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:58.717753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:58.717753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:58.717753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:58.717753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:58.717753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:58.717753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:58.717753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:58.717753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:58.717753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:58.717753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:58.717753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:58.717753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:58.719305 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:19:58.719305 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:19:58.719305 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:19:58.719305 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:58.728202 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 456/889] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:19:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:58.731171 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17913e3b-0f27-4944-be31-c079d9956f9b None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:58.745218 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b41cc4-8f96-4c6b-8328-b33b5f2a0784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:58.747361 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b41cc4-8f96-4c6b-8328-b33b5f2a0784 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:58.747925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b41cc4-8f96-4c6b-8328-b33b5f2a0784 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:58.748450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b41cc4-8f96-4c6b-8328-b33b5f2a0784 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:58.749453 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b41cc4-8f96-4c6b-8328-b33b5f2a0784 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:58.750725 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b41cc4-8f96-4c6b-8328-b33b5f2a0784 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:58.753002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b41cc4-8f96-4c6b-8328-b33b5f2a0784 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:58.759499 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 434/890] 10.222.0.121 () {56 vars in 1119 bytes} [Thu Aug 4 10:19:58 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:58.766927 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b41cc4-8f96-4c6b-8328-b33b5f2a0784 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:58.880692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:58.887927 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17913e3b-0f27-4944-be31-c079d9956f9b None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:58.918901 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b41cc4-8f96-4c6b-8328-b33b5f2a0784 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:58.940081 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c94740-2a7d-4307-909b-9bc2e713b7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:58.941903 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c94740-2a7d-4307-909b-9bc2e713b7bd None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:58.942415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c94740-2a7d-4307-909b-9bc2e713b7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:58.943229 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c94740-2a7d-4307-909b-9bc2e713b7bd None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:58.944204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c94740-2a7d-4307-909b-9bc2e713b7bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:58.944721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c94740-2a7d-4307-909b-9bc2e713b7bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:58.946650 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c94740-2a7d-4307-909b-9bc2e713b7bd None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:58.969837 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 457/891] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:19:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:58.972189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c94740-2a7d-4307-909b-9bc2e713b7bd None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:58.984798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3dae50-c4f0-49a1-a298-a9c6d5446cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:58.989660 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3dae50-c4f0-49a1-a298-a9c6d5446cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:58.990424 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3dae50-c4f0-49a1-a298-a9c6d5446cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:58.991142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3dae50-c4f0-49a1-a298-a9c6d5446cb6 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:58.992904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3dae50-c4f0-49a1-a298-a9c6d5446cb6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:58.993810 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3dae50-c4f0-49a1-a298-a9c6d5446cb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:58.995488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3dae50-c4f0-49a1-a298-a9c6d5446cb6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:59.003584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 435/892] 10.222.0.121 () {56 vars in 1119 bytes} [Thu Aug 4 10:19:58 2022] GET /identity/v3/services?type=volumev3 => generated 364 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:59.013038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3dae50-c4f0-49a1-a298-a9c6d5446cb6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:59.069623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:59.133926 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c94740-2a7d-4307-909b-9bc2e713b7bd None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:59.170678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3dae50-c4f0-49a1-a298-a9c6d5446cb6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:59.172676 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49565786-5db2-4d48-82f2-2ec300a93fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:59.173444 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49565786-5db2-4d48-82f2-2ec300a93fac None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:59.173677 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49565786-5db2-4d48-82f2-2ec300a93fac None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:59.173921 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49565786-5db2-4d48-82f2-2ec300a93fac None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:59.182750 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49565786-5db2-4d48-82f2-2ec300a93fac None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:59.184342 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49565786-5db2-4d48-82f2-2ec300a93fac None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:59.198340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-49565786-5db2-4d48-82f2-2ec300a93fac None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 10:19:59.198340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:59.198340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:59.198340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:59.198340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:59.198340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:59.198340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:59.198340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:59.198340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:59.198340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:59.198340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:59.198340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:59.198340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:19:59.198340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:19:59.198340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:19:59.198340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:19:59.199473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:59.199473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:59.199473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:19:59.199473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:59.199473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:19:59.199473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:19:59.199473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:59.199473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:59.199473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:59.199473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:59.199473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:59.199473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:59.199473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:59.199473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:59.199473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:59.199473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:59.199473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:59.199473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:59.200495 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:59.200495 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:59.200495 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:19:59.200495 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:19:59.200495 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:19:59.200495 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:19:59.200495 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:19:59.200495 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:19:59.200495 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 10:19:59.200495 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:59.204586 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 458/893] 10.222.0.121 () {56 vars in 1122 bytes} [Thu Aug 4 10:19:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:19:59.205845 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49565786-5db2-4d48-82f2-2ec300a93fac None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:59.220868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24f0e0-18c3-42e8-9321-4746f220a2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:59.221900 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24f0e0-18c3-42e8-9321-4746f220a2d5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:59.222386 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24f0e0-18c3-42e8-9321-4746f220a2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:59.222831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24f0e0-18c3-42e8-9321-4746f220a2d5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:59.223837 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c24f0e0-18c3-42e8-9321-4746f220a2d5 None admin] RBAC: Adding query filter params (interface=public, service_id=acc6149a4ac84872b085e6fe372781aa, region_id=RegionOne) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:59.224472 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c24f0e0-18c3-42e8-9321-4746f220a2d5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:59.226120 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c24f0e0-18c3-42e8-9321-4746f220a2d5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:59.229637 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:59.238102 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 436/894] 10.222.0.121 () {56 vars in 1256 bytes} [Thu Aug 4 10:19:59 2022] GET /identity/v3/endpoints?service_id=acc6149a4ac84872b085e6fe372781aa&interface=public®ion_id=RegionOne => generated 195 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:19:59.239134 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24f0e0-18c3-42e8-9321-4746f220a2d5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:59.356288 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49565786-5db2-4d48-82f2-2ec300a93fac None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:59.369082 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:59.384625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24f0e0-18c3-42e8-9321-4746f220a2d5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:59.416226 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f699d5a-db93-4f23-b5d5-3a28389f94de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:59.417912 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f699d5a-db93-4f23-b5d5-3a28389f94de None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:59.418642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f699d5a-db93-4f23-b5d5-3a28389f94de None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:59.419409 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f699d5a-db93-4f23-b5d5-3a28389f94de None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:59.422736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f699d5a-db93-4f23-b5d5-3a28389f94de None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:19:59.423625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f699d5a-db93-4f23-b5d5-3a28389f94de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:59.426581 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f699d5a-db93-4f23-b5d5-3a28389f94de None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:59.434397 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7730b-913a-4693-b104-b330f3ac7b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:59.435044 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7730b-913a-4693-b104-b330f3ac7b12 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:59.435316 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7730b-913a-4693-b104-b330f3ac7b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:59.435636 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7730b-913a-4693-b104-b330f3ac7b12 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:59.436357 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7730b-913a-4693-b104-b330f3ac7b12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:59.437825 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7730b-913a-4693-b104-b330f3ac7b12 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:59.449938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 437/895] 10.222.0.121 () {56 vars in 1092 bytes} [Thu Aug 4 10:19:59 2022] GET /identity/v3/services => generated 1274 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:19:59.467267 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 459/896] 10.222.0.121 () {56 vars in 1154 bytes} [Thu Aug 4 10:19:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:19:59.477275 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7730b-913a-4693-b104-b330f3ac7b12 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:59.513733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d5579-c1d1-4727-b36d-9c72ec169407 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:59.514480 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d5579-c1d1-4727-b36d-9c72ec169407 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:59.514984 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d5579-c1d1-4727-b36d-9c72ec169407 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:59.517213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 460/897] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:19:59 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:19:59.528140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4201ef-daba-4329-b54c-cc5bdfc5ef52 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:59.528943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4201ef-daba-4329-b54c-cc5bdfc5ef52 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:59.533805 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4201ef-daba-4329-b54c-cc5bdfc5ef52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:59.562644 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-7d4201ef-daba-4329-b54c-cc5bdfc5ef52 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:19:59.564729 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-7d4201ef-daba-4329-b54c-cc5bdfc5ef52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:19:59.658542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 461/898] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:19:59 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:19:59.676764 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7730b-913a-4693-b104-b330f3ac7b12 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:59.693204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4201ef-daba-4329-b54c-cc5bdfc5ef52 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:19:59.704329 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:59.731619 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2ed8f-2469-48b5-9a2f-77f261abef74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:59.732670 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2ed8f-2469-48b5-9a2f-77f261abef74 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:59.732670 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2ed8f-2469-48b5-9a2f-77f261abef74 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:59.732837 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2ed8f-2469-48b5-9a2f-77f261abef74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:59.763587 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:19:59.835017 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c2ed8f-2469-48b5-9a2f-77f261abef74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:59.837326 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c2ed8f-2469-48b5-9a2f-77f261abef74 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:59.889197 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4201ef-daba-4329-b54c-cc5bdfc5ef52 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:19:59.935567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6728e5e1-8ea9-4268-a721-90a6367df3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:19:59.936353 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728e5e1-8ea9-4268-a721-90a6367df3b5 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:19:59.936737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728e5e1-8ea9-4268-a721-90a6367df3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:19:59.937116 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728e5e1-8ea9-4268-a721-90a6367df3b5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:19:59.937875 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6728e5e1-8ea9-4268-a721-90a6367df3b5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:19:59.939444 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6728e5e1-8ea9-4268-a721-90a6367df3b5 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:19:59.951384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-6728e5e1-8ea9-4268-a721-90a6367df3b5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 10:19:59.951384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:59.951384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:19:59.951384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:19:59.951384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:19:59.951384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:19:59.951384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:19:59.951384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:19:59.951384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 10:19:59.951384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:59.951384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:19:59.951384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:59.951384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:19:59.951384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:19:59.951384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:19:59.951384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:19:59.954062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:19:59.954062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:19:59.954062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:19:59.954062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:19:59.954062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:19:59.954062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:19:59.954062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:19:59.954062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:19:59.954062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:19:59.954062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:19:59.954062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:19:59.954062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:19:59.954062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:19:59.954062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:19:59.954062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:19:59.954062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:19:59.955577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:19:59.955577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:19:59.955577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:19:59.955577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:19:59.955577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:19:59.955577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:19:59.955577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:19:59.955577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:19:59.955577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:19:59.955577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:19:59.955577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:19:59.955577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:19:59.955577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:19:59.955577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 10:19:59.955577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:19:59.957350 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 462/899] 10.222.0.121 () {56 vars in 1137 bytes} [Thu Aug 4 10:19:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:19:59.959853 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6728e5e1-8ea9-4268-a721-90a6367df3b5 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:00.011635 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 438/900] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:19:59 2022] GET /identity/v3/auth/tokens => generated 1374 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 10:20:00.017747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2ed8f-2469-48b5-9a2f-77f261abef74 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:00.055674 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:00.098265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6728e5e1-8ea9-4268-a721-90a6367df3b5 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:00.112409 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:00.150686 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be8104-7536-4ff7-a64d-06ba1b8001c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:00.151715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be8104-7536-4ff7-a64d-06ba1b8001c0 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:00.152214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be8104-7536-4ff7-a64d-06ba1b8001c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:00.152702 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be8104-7536-4ff7-a64d-06ba1b8001c0 None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:00.153111 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2ed8f-2469-48b5-9a2f-77f261abef74 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:00.153810 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be8104-7536-4ff7-a64d-06ba1b8001c0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:00.154306 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be8104-7536-4ff7-a64d-06ba1b8001c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:00.156365 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be8104-7536-4ff7-a64d-06ba1b8001c0 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:00.162744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 463/901] 10.222.0.121 () {56 vars in 1131 bytes} [Thu Aug 4 10:19:59 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:00.168917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c53ef-f148-499e-b306-36046e67c919 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:00.169394 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c53ef-f148-499e-b306-36046e67c919 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:00.169599 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c53ef-f148-499e-b306-36046e67c919 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:00.172297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 464/902] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:00 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:00.172579 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c53ef-f148-499e-b306-36046e67c919 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:00.198035 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4774ffc-3ab1-4f65-8ec8-a184e43ef8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:00.198590 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4774ffc-3ab1-4f65-8ec8-a184e43ef8e8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:00.198867 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4774ffc-3ab1-4f65-8ec8-a184e43ef8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:00.199152 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4774ffc-3ab1-4f65-8ec8-a184e43ef8e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:00.210463 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4774ffc-3ab1-4f65-8ec8-a184e43ef8e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:00.211994 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4774ffc-3ab1-4f65-8ec8-a184e43ef8e8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:00.229057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-a4774ffc-3ab1-4f65-8ec8-a184e43ef8e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:20:00.229057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:00.229057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:00.229057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:00.229057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:00.229057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:00.229057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:00.229057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:00.229057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:00.229057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:00.229057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:00.229057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:00.229057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:20:00.229057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:20:00.229057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:20:00.230125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:20:00.230125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:00.230125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:00.230125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:00.230125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:00.230125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:00.230125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:00.230125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:00.230125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:00.230125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:00.230125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:00.230125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:00.230125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:00.230125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:00.230125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:00.230125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:00.230125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:00.231063 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:20:00.231063 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:20:00.231063 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:20:00.231063 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:00.231616 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 439/903] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:20:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:00.233484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914379f-2cfc-4b1d-b1f3-2f1809e8b897 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:00.233981 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914379f-2cfc-4b1d-b1f3-2f1809e8b897 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:00.234473 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914379f-2cfc-4b1d-b1f3-2f1809e8b897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:00.265594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4914379f-2cfc-4b1d-b1f3-2f1809e8b897 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:00.267677 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4914379f-2cfc-4b1d-b1f3-2f1809e8b897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:00.310762 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c53ef-f148-499e-b306-36046e67c919 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:00.369872 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf7dfa-e0bc-48e6-8739-cd1c8a4a7477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:00.370522 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf7dfa-e0bc-48e6-8739-cd1c8a4a7477 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:00.370842 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf7dfa-e0bc-48e6-8739-cd1c8a4a7477 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:00.371139 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf7dfa-e0bc-48e6-8739-cd1c8a4a7477 None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:00.371778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf7dfa-e0bc-48e6-8739-cd1c8a4a7477 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:00.372103 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf7dfa-e0bc-48e6-8739-cd1c8a4a7477 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:00.373696 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf7dfa-e0bc-48e6-8739-cd1c8a4a7477 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:00.375528 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 440/904] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:00 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:00.376123 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:00.376270 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914379f-2cfc-4b1d-b1f3-2f1809e8b897 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:00.387196 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 465/905] 10.222.0.121 () {56 vars in 1131 bytes} [Thu Aug 4 10:20:00 2022] GET /identity/v3/services?type=compute_legacy => generated 388 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:00.397711 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf7dfa-e0bc-48e6-8739-cd1c8a4a7477 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:00.432779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:00.534224 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914379f-2cfc-4b1d-b1f3-2f1809e8b897 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:00.604222 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac52f4-e579-417a-a38c-06a45f6a4ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:00.605478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac52f4-e579-417a-a38c-06a45f6a4ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:00.605478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac52f4-e579-417a-a38c-06a45f6a4ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:00.605478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac52f4-e579-417a-a38c-06a45f6a4ba4 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:00.605739 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac52f4-e579-417a-a38c-06a45f6a4ba4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:00.606099 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac52f4-e579-417a-a38c-06a45f6a4ba4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:00.607544 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac52f4-e579-417a-a38c-06a45f6a4ba4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:00.613647 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf7dfa-e0bc-48e6-8739-cd1c8a4a7477 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:00.630919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 441/906] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:20:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:00.631762 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac52f4-e579-417a-a38c-06a45f6a4ba4 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:00.667735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef819323-a5fa-4538-9129-3fe889ef6b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:00.668523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef819323-a5fa-4538-9129-3fe889ef6b35 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:00.668813 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef819323-a5fa-4538-9129-3fe889ef6b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:00.669103 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef819323-a5fa-4538-9129-3fe889ef6b35 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:00.669937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef819323-a5fa-4538-9129-3fe889ef6b35 None admin] RBAC: Adding query filter params (interface=public, service_id=ef0e7e081e1a49d3a52e1a5af7dbf8ab, region_id=RegionOne) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:00.670269 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef819323-a5fa-4538-9129-3fe889ef6b35 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:00.671887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef819323-a5fa-4538-9129-3fe889ef6b35 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:00.681656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 466/907] 10.222.0.121 () {56 vars in 1256 bytes} [Thu Aug 4 10:20:00 2022] GET /identity/v3/endpoints?service_id=ef0e7e081e1a49d3a52e1a5af7dbf8ab&interface=public®ion_id=RegionOne => generated 195 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:00.687803 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef819323-a5fa-4538-9129-3fe889ef6b35 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:00.751316 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:00.760262 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac52f4-e579-417a-a38c-06a45f6a4ba4 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:00.790904 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:00.807838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6841f38-902c-4fd7-aad8-a68a6cd31954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:00.808522 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6841f38-902c-4fd7-aad8-a68a6cd31954 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:00.808804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6841f38-902c-4fd7-aad8-a68a6cd31954 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:00.809081 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6841f38-902c-4fd7-aad8-a68a6cd31954 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:00.809799 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6841f38-902c-4fd7-aad8-a68a6cd31954 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:00.811202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6841f38-902c-4fd7-aad8-a68a6cd31954 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:00.828378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 442/908] 10.222.0.121 () {60 vars in 1146 bytes} [Thu Aug 4 10:20:00 2022] POST /identity/v3/services => generated 242 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:20:00.829255 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6841f38-902c-4fd7-aad8-a68a6cd31954 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:00.854484 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef819323-a5fa-4538-9129-3fe889ef6b35 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:00.918617 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f872a-10f5-428d-b4ad-64d7cc4285d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:00.919424 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f872a-10f5-428d-b4ad-64d7cc4285d2 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:00.919727 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f872a-10f5-428d-b4ad-64d7cc4285d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:00.920027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f872a-10f5-428d-b4ad-64d7cc4285d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:00.931907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f872a-10f5-428d-b4ad-64d7cc4285d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:00.933647 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f872a-10f5-428d-b4ad-64d7cc4285d2 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:00.950723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-6c4f872a-10f5-428d-b4ad-64d7cc4285d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:20:00.950723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:00.950723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:00.950723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:00.950723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:00.950723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:00.950723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:00.950723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:00.950723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:00.950723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:00.950723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:00.950723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:00.950723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:20:00.950723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:20:00.950723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:20:00.952991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:20:00.952991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:00.952991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:00.952991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:00.952991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:00.952991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:00.952991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:00.952991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:00.952991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:00.952991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:00.952991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:00.952991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:00.952991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:00.952991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:00.952991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:00.952991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:00.952991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:00.953972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:20:00.953972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:20:00.953972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:20:00.953972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:20:00.953972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:20:00.953972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:20:00.953972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:20:00.953972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:00.953972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 467/909] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:20:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:00.960195 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f872a-10f5-428d-b4ad-64d7cc4285d2 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:00.989550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6841f38-902c-4fd7-aad8-a68a6cd31954 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:01.035797 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907ca47-95ac-42e8-915d-4fc36da0a714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:01.037569 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907ca47-95ac-42e8-915d-4fc36da0a714 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:01.037930 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907ca47-95ac-42e8-915d-4fc36da0a714 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:01.038210 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907ca47-95ac-42e8-915d-4fc36da0a714 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:01.038947 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907ca47-95ac-42e8-915d-4fc36da0a714 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:01.040585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907ca47-95ac-42e8-915d-4fc36da0a714 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:01.047882 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 443/910] 10.222.0.121 () {56 vars in 1092 bytes} [Thu Aug 4 10:20:00 2022] GET /identity/v3/services => generated 1504 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:20:01.117573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f872a-10f5-428d-b4ad-64d7cc4285d2 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:01.156400 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:01.167811 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377b1ac-6120-4c18-8493-4fc6839b04cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:01.169196 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377b1ac-6120-4c18-8493-4fc6839b04cf None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:01.169863 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377b1ac-6120-4c18-8493-4fc6839b04cf None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:01.170591 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377b1ac-6120-4c18-8493-4fc6839b04cf None admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:01.171618 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d377b1ac-6120-4c18-8493-4fc6839b04cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:01.172195 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d377b1ac-6120-4c18-8493-4fc6839b04cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:01.173922 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d377b1ac-6120-4c18-8493-4fc6839b04cf None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:01.190752 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 468/911] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:20:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:01.196959 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907ca47-95ac-42e8-915d-4fc36da0a714 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:01.222894 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:01.334263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907ca47-95ac-42e8-915d-4fc36da0a714 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:01.375778 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055f871-cb13-460f-be33-176a332bd9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:01.376529 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055f871-cb13-460f-be33-176a332bd9fd None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:01.376823 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055f871-cb13-460f-be33-176a332bd9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:01.377182 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055f871-cb13-460f-be33-176a332bd9fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:01.378005 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055f871-cb13-460f-be33-176a332bd9fd None admin] RBAC: Adding query filter params (role.id=9395de2ca6f34dfc85ea530fc0a14f45, scope.project.id=22768aba760b4307aab131d1312b2bf6, user.id=efe6e59228bd49bfbff25a11eb682426) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:01.378328 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055f871-cb13-460f-be33-176a332bd9fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:01.379838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055f871-cb13-460f-be33-176a332bd9fd None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:01.387646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 444/912] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:20:01 2022] GET /identity/v3/role_assignments?user.id=efe6e59228bd49bfbff25a11eb682426&scope.project.id=22768aba760b4307aab131d1312b2bf6&role.id=9395de2ca6f34dfc85ea530fc0a14f45 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:01.681887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a1390-2401-4141-b1a5-7afce6e8c901 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:01.682110 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a1390-2401-4141-b1a5-7afce6e8c901 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:01.682337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a1390-2401-4141-b1a5-7afce6e8c901 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:01.683756 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 469/913] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:01.689863 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ccd5d-b761-4c60-aaa3-e6dc9e4a06a8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:01.690152 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ccd5d-b761-4c60-aaa3-e6dc9e4a06a8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:01.690438 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ccd5d-b761-4c60-aaa3-e6dc9e4a06a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:01.710802 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-bb0ccd5d-b761-4c60-aaa3-e6dc9e4a06a8 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:01.712093 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bb0ccd5d-b761-4c60-aaa3-e6dc9e4a06a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:01.788827 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 445/914] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:01 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:01.812320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a1390-2401-4141-b1a5-7afce6e8c901 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:01.928381 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a1390-2401-4141-b1a5-7afce6e8c901 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:01.968201 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c04993-6524-4d89-9b7a-dd1f9a1a5659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:01.968943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c04993-6524-4d89-9b7a-dd1f9a1a5659 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:01.969026 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c04993-6524-4d89-9b7a-dd1f9a1a5659 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:01.969309 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c04993-6524-4d89-9b7a-dd1f9a1a5659 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:01.970027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c04993-6524-4d89-9b7a-dd1f9a1a5659 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:01.971690 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c04993-6524-4d89-9b7a-dd1f9a1a5659 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:01.982740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-44c04993-6524-4d89-9b7a-dd1f9a1a5659 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 10:20:01.982740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:01.982740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:20:01.982740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:20:01.982740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:20:01.982740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:20:01.982740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:20:01.982740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:20:01.982740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 10:20:01.982740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:01.982740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:20:01.982740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:01.982740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:01.982740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:01.982740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:01.982740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:01.982740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:01.983909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:01.983909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:01.983909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:01.983909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:01.983909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:01.983909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:01.983909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:20:01.983909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:20:01.983909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:20:01.983909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:20:01.983909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:01.983909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:01.983909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:01.983909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:01.983909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:01.983909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:01.984868 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:01.984868 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:01.984868 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:01.984868 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:01.984868 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:01.984868 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:01.984868 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:01.984868 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:01.984868 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:01.984868 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:01.984868 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:20:01.984868 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:20:01.984868 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 10:20:01.984868 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:01.984868 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 470/915] 10.222.0.121 () {56 vars in 1119 bytes} [Thu Aug 4 10:20:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:01.991825 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ccd5d-b761-4c60-aaa3-e6dc9e4a06a8 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:02.007780 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:02.108463 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ccd5d-b761-4c60-aaa3-e6dc9e4a06a8 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:02.154157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9bca8-aaaa-4e54-8d1d-7099ed0b6a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:02.154915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9bca8-aaaa-4e54-8d1d-7099ed0b6a57 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:02.158430 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9bca8-aaaa-4e54-8d1d-7099ed0b6a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:02.158430 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9bca8-aaaa-4e54-8d1d-7099ed0b6a57 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:02.158430 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b9bca8-aaaa-4e54-8d1d-7099ed0b6a57 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:02.158430 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b9bca8-aaaa-4e54-8d1d-7099ed0b6a57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:02.158430 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b9bca8-aaaa-4e54-8d1d-7099ed0b6a57 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:02.164664 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 446/916] 10.222.0.121 () {56 vars in 1119 bytes} [Thu Aug 4 10:20:01 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:02.170383 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c04993-6524-4d89-9b7a-dd1f9a1a5659 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:02.227468 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:02.335030 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c04993-6524-4d89-9b7a-dd1f9a1a5659 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:02.398450 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd870459-e899-49bf-b7cb-8dc771ae7d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:02.400152 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd870459-e899-49bf-b7cb-8dc771ae7d45 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:02.400935 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd870459-e899-49bf-b7cb-8dc771ae7d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:02.401707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd870459-e899-49bf-b7cb-8dc771ae7d45 None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:02.403175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd870459-e899-49bf-b7cb-8dc771ae7d45 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:02.404061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd870459-e899-49bf-b7cb-8dc771ae7d45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:02.406942 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd870459-e899-49bf-b7cb-8dc771ae7d45 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:02.417677 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 471/917] 10.222.0.121 () {56 vars in 1119 bytes} [Thu Aug 4 10:20:02 2022] GET /identity/v3/services?type=volumev3 => generated 364 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:02.423303 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9bca8-aaaa-4e54-8d1d-7099ed0b6a57 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:02.507266 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:02.535111 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9bca8-aaaa-4e54-8d1d-7099ed0b6a57 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:02.579836 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcba78b-ed71-40a0-9184-cc03358c7a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:02.580733 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcba78b-ed71-40a0-9184-cc03358c7a8b None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:02.581026 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcba78b-ed71-40a0-9184-cc03358c7a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:02.581305 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcba78b-ed71-40a0-9184-cc03358c7a8b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:02.582075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcba78b-ed71-40a0-9184-cc03358c7a8b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:02.583541 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcba78b-ed71-40a0-9184-cc03358c7a8b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:02.612098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 447/918] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:20:02 2022] POST /identity/v3/endpoints => generated 348 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:20:03.158502 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1a22d-5215-48f7-b3e1-dd9dc3975ed2 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:03.159545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1a22d-5215-48f7-b3e1-dd9dc3975ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:03.160007 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1a22d-5215-48f7-b3e1-dd9dc3975ed2 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:03.162028 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 472/919] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:03 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:03.168358 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6823830-b70a-442c-ac97-38371f527cdf None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:03.169399 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6823830-b70a-442c-ac97-38371f527cdf None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:03.169886 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6823830-b70a-442c-ac97-38371f527cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:03.191631 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d6823830-b70a-442c-ac97-38371f527cdf None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:03.193534 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d6823830-b70a-442c-ac97-38371f527cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:03.286212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 448/920] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:03 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:03.313710 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f1a22d-5215-48f7-b3e1-dd9dc3975ed2 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:03.325329 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355a2b4-26e8-4d7a-ac14-131e98326ca3 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:03.325774 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355a2b4-26e8-4d7a-ac14-131e98326ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:03.326266 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355a2b4-26e8-4d7a-ac14-131e98326ca3 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:03.329091 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 449/921] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:03 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:03.337397 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272cee3-cddb-4eba-9c83-abf97ce90748 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:03.337604 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272cee3-cddb-4eba-9c83-abf97ce90748 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:03.337857 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272cee3-cddb-4eba-9c83-abf97ce90748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:03.364255 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2272cee3-cddb-4eba-9c83-abf97ce90748 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:03.366180 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2272cee3-cddb-4eba-9c83-abf97ce90748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:03.435867 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f1a22d-5215-48f7-b3e1-dd9dc3975ed2 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:03.444388 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:03.462589 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:03.478016 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccea0e0-6e1b-46aa-802e-87a15aa2ff14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:03.478885 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccea0e0-6e1b-46aa-802e-87a15aa2ff14 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:03.479408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccea0e0-6e1b-46aa-802e-87a15aa2ff14 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:03.480749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccea0e0-6e1b-46aa-802e-87a15aa2ff14 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:03.480749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccea0e0-6e1b-46aa-802e-87a15aa2ff14 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:03.488479 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccea0e0-6e1b-46aa-802e-87a15aa2ff14 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:03.488785 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 450/922] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:03 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:03.492913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-4ccea0e0-6e1b-46aa-802e-87a15aa2ff14 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:20:03.492913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:03.492913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:20:03.492913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:20:03.492913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:20:03.492913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:20:03.492913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:20:03.492913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:20:03.492913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:20:03.492913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:03.492913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:20:03.492913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:03.492913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:03.492913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:03.492913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:03.492913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:03.492913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:03.494217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:03.494217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:03.494217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:03.494217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:03.494217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:03.494217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:03.494217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:20:03.494217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:20:03.494217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:20:03.494217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:20:03.494217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:03.494217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:03.494217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:03.494217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:03.494217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:03.494217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:03.495308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:03.495308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:03.495308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:03.495308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:03.495308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:03.495308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:03.495308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:03.495308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:03.495308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:03.495308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:03.495308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:20:03.495308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:20:03.495308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:20:03.495308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:03.497639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 473/923] 10.222.0.121 () {56 vars in 1131 bytes} [Thu Aug 4 10:20:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:03.505341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272cee3-cddb-4eba-9c83-abf97ce90748 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:03.511548 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccea0e0-6e1b-46aa-802e-87a15aa2ff14 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:03.623992 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccea0e0-6e1b-46aa-802e-87a15aa2ff14 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:03.627361 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272cee3-cddb-4eba-9c83-abf97ce90748 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:03.672666 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859cb0f0-0795-48c0-a77b-ab853f10369b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:03.673471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cb0f0-0795-48c0-a77b-ab853f10369b None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:03.673761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cb0f0-0795-48c0-a77b-ab853f10369b None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:03.673880 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e28636-8c27-4490-8972-4801c1a66a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:03.674070 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cb0f0-0795-48c0-a77b-ab853f10369b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:03.674551 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e28636-8c27-4490-8972-4801c1a66a87 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:03.674744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859cb0f0-0795-48c0-a77b-ab853f10369b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:03.674947 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e28636-8c27-4490-8972-4801c1a66a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:03.675233 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e28636-8c27-4490-8972-4801c1a66a87 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:03.675985 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e28636-8c27-4490-8972-4801c1a66a87 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:03.676188 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859cb0f0-0795-48c0-a77b-ab853f10369b None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:03.676287 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e28636-8c27-4490-8972-4801c1a66a87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:03.677743 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e28636-8c27-4490-8972-4801c1a66a87 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:03.692428 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 451/924] 10.222.0.121 () {56 vars in 1127 bytes} [Thu Aug 4 10:20:03 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:03.692428 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d50a5-7a72-4116-9486-8eaa033f879e None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:03.692428 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d50a5-7a72-4116-9486-8eaa033f879e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:03.692428 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d50a5-7a72-4116-9486-8eaa033f879e None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:03.693061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-859cb0f0-0795-48c0-a77b-ab853f10369b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 10:20:03.693061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:03.693061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:20:03.693061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:20:03.693061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:20:03.693061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:20:03.693061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:20:03.693061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:20:03.693061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 10:20:03.693061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:03.693061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:20:03.693061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:03.693061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:03.693061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:03.693061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:03.693061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:03.694555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:03.694555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:03.694555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:03.694555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:03.694555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:03.694555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:03.694555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:03.694555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:20:03.694555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:20:03.694555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:20:03.694555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:20:03.694555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:03.694555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:03.694555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:03.694555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:03.694555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:03.698595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:03.698595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:03.698595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:03.698595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:03.698595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:03.698595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:03.698595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:03.698595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:03.698595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:03.698595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:03.698595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:03.698595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:20:03.698595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:20:03.698595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 10:20:03.698595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:03.699465 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 452/925] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:03 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:03.699465 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f7a2f-8409-4d94-a311-51baa3df0f36 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:03.699465 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f7a2f-8409-4d94-a311-51baa3df0f36 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:03.699465 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f7a2f-8409-4d94-a311-51baa3df0f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:03.713435 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 474/926] 10.222.0.121 () {56 vars in 1137 bytes} [Thu Aug 4 10:20:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:20:03.713435 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859cb0f0-0795-48c0-a77b-ab853f10369b None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:03.719115 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-431f7a2f-8409-4d94-a311-51baa3df0f36 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:03.720490 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-431f7a2f-8409-4d94-a311-51baa3df0f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:03.755202 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:03.781835 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:03.818651 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 453/927] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:03 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:03.820912 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f7a2f-8409-4d94-a311-51baa3df0f36 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:03.861671 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859cb0f0-0795-48c0-a77b-ab853f10369b None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:03.911293 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56acea84-3edd-4110-aacb-0a3e60ded3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:03.912628 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acea84-3edd-4110-aacb-0a3e60ded3a1 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:03.913008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acea84-3edd-4110-aacb-0a3e60ded3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:03.913501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acea84-3edd-4110-aacb-0a3e60ded3a1 None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:03.914808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56acea84-3edd-4110-aacb-0a3e60ded3a1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:03.915286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56acea84-3edd-4110-aacb-0a3e60ded3a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:03.918152 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56acea84-3edd-4110-aacb-0a3e60ded3a1 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:03.932646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 475/928] 10.222.0.121 () {56 vars in 1127 bytes} [Thu Aug 4 10:20:03 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:03.932646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56acea84-3edd-4110-aacb-0a3e60ded3a1 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:03.965140 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f7a2f-8409-4d94-a311-51baa3df0f36 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:04.021026 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108afd23-c275-4b1a-aaf9-715ea3e16172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:04.022221 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108afd23-c275-4b1a-aaf9-715ea3e16172 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:04.022791 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108afd23-c275-4b1a-aaf9-715ea3e16172 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:04.023281 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108afd23-c275-4b1a-aaf9-715ea3e16172 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:04.024517 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108afd23-c275-4b1a-aaf9-715ea3e16172 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:04.025092 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108afd23-c275-4b1a-aaf9-715ea3e16172 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:04.027667 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108afd23-c275-4b1a-aaf9-715ea3e16172 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:04.036926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 454/929] 10.222.0.121 () {56 vars in 1131 bytes} [Thu Aug 4 10:20:03 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:04.038399 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108afd23-c275-4b1a-aaf9-715ea3e16172 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:04.095502 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:04.100982 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56acea84-3edd-4110-aacb-0a3e60ded3a1 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:04.107256 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:04.150220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f9b37-b087-4e3b-a647-1ae5e69c6c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:04.150938 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f9b37-b087-4e3b-a647-1ae5e69c6c92 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:04.151474 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f9b37-b087-4e3b-a647-1ae5e69c6c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:04.151474 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f9b37-b087-4e3b-a647-1ae5e69c6c92 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:04.163896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f9b37-b087-4e3b-a647-1ae5e69c6c92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:20:04.169404 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-8c7f9b37-b087-4e3b-a647-1ae5e69c6c92 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 10:20:04.169404 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:04.169404 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:04.169404 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:04.169404 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:04.169404 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:04.169404 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:04.169404 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:04.169404 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:04.169404 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:04.169404 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:04.169404 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:04.169404 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:20:04.169404 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:20:04.169404 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:20:04.169404 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:20:04.170412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:20:04.170412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:20:04.170412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:20:04.170412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:20:04.170412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:04.170412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:04.170412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:04.170412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:04.170412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:04.170412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:04.170412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:04.170412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:04.170412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:04.170412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:04.170412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:04.170412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:04.171313 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:04.171313 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:04.171313 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:04.171313 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:04.171313 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:20:04.171313 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:20:04.171313 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:20:04.171313 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:20:04.171313 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:20:04.171313 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:20:04.171313 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 10:20:04.171313 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:04.171313 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 476/930] 10.222.0.121 () {56 vars in 1125 bytes} [Thu Aug 4 10:20:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:04.173214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f9b37-b087-4e3b-a647-1ae5e69c6c92 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:04.191141 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108afd23-c275-4b1a-aaf9-715ea3e16172 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:04.234074 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1082425-50d8-4a35-91e6-fef912547a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:04.234751 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1082425-50d8-4a35-91e6-fef912547a37 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:04.235091 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1082425-50d8-4a35-91e6-fef912547a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:04.235431 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1082425-50d8-4a35-91e6-fef912547a37 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:04.236325 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1082425-50d8-4a35-91e6-fef912547a37 None admin] RBAC: Adding query filter params (interface=public, service_id=e89f4b14b48c40a199cd99daa6b0d0f6, region_id=RegionOne) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:04.236710 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1082425-50d8-4a35-91e6-fef912547a37 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:04.238568 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1082425-50d8-4a35-91e6-fef912547a37 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:04.251129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 455/931] 10.222.0.121 () {56 vars in 1256 bytes} [Thu Aug 4 10:20:04 2022] GET /identity/v3/endpoints?service_id=e89f4b14b48c40a199cd99daa6b0d0f6&interface=public®ion_id=RegionOne => generated 195 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:04.251129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1082425-50d8-4a35-91e6-fef912547a37 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:04.315223 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f9b37-b087-4e3b-a647-1ae5e69c6c92 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:04.356210 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d1a31-2019-41c3-a6aa-52551fa3ea7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:04.358466 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d1a31-2019-41c3-a6aa-52551fa3ea7c None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:04.359269 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d1a31-2019-41c3-a6aa-52551fa3ea7c None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:04.359552 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d1a31-2019-41c3-a6aa-52551fa3ea7c None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:04.360300 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d1a31-2019-41c3-a6aa-52551fa3ea7c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:04.360615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d1a31-2019-41c3-a6aa-52551fa3ea7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:04.363061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d1a31-2019-41c3-a6aa-52551fa3ea7c None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:04.380183 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 477/932] 10.222.0.121 () {56 vars in 1131 bytes} [Thu Aug 4 10:20:04 2022] GET /identity/v3/services?type=compute_legacy => generated 388 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:04.380183 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d1a31-2019-41c3-a6aa-52551fa3ea7c None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:04.408864 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:04.413781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1082425-50d8-4a35-91e6-fef912547a37 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:04.426451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:04.456097 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c5b1a-cb4a-4fcd-b627-f3381734bdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:04.456731 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c5b1a-cb4a-4fcd-b627-f3381734bdec None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:04.457005 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c5b1a-cb4a-4fcd-b627-f3381734bdec None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:04.457319 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c5b1a-cb4a-4fcd-b627-f3381734bdec None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:04.457990 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c5b1a-cb4a-4fcd-b627-f3381734bdec None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:04.458308 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c5b1a-cb4a-4fcd-b627-f3381734bdec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:04.459924 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c5b1a-cb4a-4fcd-b627-f3381734bdec None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:04.474673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 456/933] 10.222.0.121 () {56 vars in 1120 bytes} [Thu Aug 4 10:20:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:04.474673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c5b1a-cb4a-4fcd-b627-f3381734bdec None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:04.509270 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d1a31-2019-41c3-a6aa-52551fa3ea7c None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:04.559000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c089a-f37b-496c-8626-ddb229abb5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:04.559702 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c089a-f37b-496c-8626-ddb229abb5aa None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:04.560010 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c089a-f37b-496c-8626-ddb229abb5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:04.560297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c089a-f37b-496c-8626-ddb229abb5aa None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:04.561024 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c089a-f37b-496c-8626-ddb229abb5aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:04.562403 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c089a-f37b-496c-8626-ddb229abb5aa None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:04.569532 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 478/934] 10.222.0.121 () {56 vars in 1092 bytes} [Thu Aug 4 10:20:04 2022] GET /identity/v3/services => generated 1504 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:20:04.570408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c089a-f37b-496c-8626-ddb229abb5aa None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:04.657040 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c5b1a-cb4a-4fcd-b627-f3381734bdec None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:04.692730 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:04.704255 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06875624-1e76-4c1c-a9e1-175035901a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:04.705330 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06875624-1e76-4c1c-a9e1-175035901a4e None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:04.705800 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06875624-1e76-4c1c-a9e1-175035901a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:04.706306 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06875624-1e76-4c1c-a9e1-175035901a4e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:04.707066 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06875624-1e76-4c1c-a9e1-175035901a4e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:04.708537 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06875624-1e76-4c1c-a9e1-175035901a4e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:04.713695 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c089a-f37b-496c-8626-ddb229abb5aa None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:04.749872 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 457/935] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:20:04 2022] POST /identity/v3/endpoints => generated 349 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:20:04.796857 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872da664-0c39-405d-b706-0326f8b0d2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:04.797905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872da664-0c39-405d-b706-0326f8b0d2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:04.798342 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872da664-0c39-405d-b706-0326f8b0d2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:04.798628 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872da664-0c39-405d-b706-0326f8b0d2d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:04.813840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872da664-0c39-405d-b706-0326f8b0d2d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:04.815485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872da664-0c39-405d-b706-0326f8b0d2d4 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:04.834573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-872da664-0c39-405d-b706-0326f8b0d2d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:20:04.834573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:04.834573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:04.834573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:04.834573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:04.834573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:04.834573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:04.834573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:04.834573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:04.834573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:04.834573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:04.834573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:04.834573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:20:04.834573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:20:04.834573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:20:04.835687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:20:04.835687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:04.835687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:04.835687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:04.835687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:04.835687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:04.835687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:04.835687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:04.835687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:04.835687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:04.835687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:04.835687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:04.835687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:04.835687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:04.835687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:04.835687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:04.835687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:04.836729 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:20:04.836729 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:20:04.836729 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:20:04.836729 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:04.836729 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 479/936] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:20:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:04.839242 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d5e23-61c7-42fe-81a2-2329d874cb58 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:04.839537 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d5e23-61c7-42fe-81a2-2329d874cb58 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:04.839830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d5e23-61c7-42fe-81a2-2329d874cb58 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:04.841634 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 458/937] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 10:20:04.848175 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d5e23-61c7-42fe-81a2-2329d874cb58 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:04.848860 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d89d8-d613-46eb-9d14-8171b78250ac None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:04.849581 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d89d8-d613-46eb-9d14-8171b78250ac None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:04.850293 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d89d8-d613-46eb-9d14-8171b78250ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:04.877617 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-9c4d89d8-d613-46eb-9d14-8171b78250ac None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:04.879093 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-9c4d89d8-d613-46eb-9d14-8171b78250ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:04.895562 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:04.983194 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 480/938] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:04 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:05.007693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d89d8-d613-46eb-9d14-8171b78250ac None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:05.034243 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d5e23-61c7-42fe-81a2-2329d874cb58 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:05.088497 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:05.089995 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdea1d0-16d9-42cc-baf3-f9f3c3ed3ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:05.090636 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdea1d0-16d9-42cc-baf3-f9f3c3ed3ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:05.090960 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdea1d0-16d9-42cc-baf3-f9f3c3ed3ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:05.091298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdea1d0-16d9-42cc-baf3-f9f3c3ed3ef0 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:05.092084 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdea1d0-16d9-42cc-baf3-f9f3c3ed3ef0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:05.092398 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdea1d0-16d9-42cc-baf3-f9f3c3ed3ef0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:05.094807 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdea1d0-16d9-42cc-baf3-f9f3c3ed3ef0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:05.113785 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 459/939] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:20:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:05.120372 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdea1d0-16d9-42cc-baf3-f9f3c3ed3ef0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:05.143350 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d89d8-d613-46eb-9d14-8171b78250ac None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:05.195830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b511957-d032-44bf-98a3-053ea515ee09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:05.196746 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b511957-d032-44bf-98a3-053ea515ee09 None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:05.197037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b511957-d032-44bf-98a3-053ea515ee09 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:05.199515 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b511957-d032-44bf-98a3-053ea515ee09 None admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:05.199515 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b511957-d032-44bf-98a3-053ea515ee09 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:05.200689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b511957-d032-44bf-98a3-053ea515ee09 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:05.205230 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: WARNING keystone.server.flask.common [None req-2b511957-d032-44bf-98a3-053ea515ee09 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 04 10:20:05.235161 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:05.238162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.sql.core [None req-2b511957-d032-44bf-98a3-053ea515ee09 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 10:20:05.238162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 04 10:20:05.238162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [parameters: {'id': 'df551692cecd4fd2944e68d1d64b6b34', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 04 10:20:05.238162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80235) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 10:20:05.250638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-2b511957-d032-44bf-98a3-053ea515ee09 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id df551692cecd4fd2944e68d1d64b6b34.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id df551692cecd4fd2944e68d1d64b6b34. Aug 04 10:20:05.250638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:05.250638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 10:20:05.250638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 10:20:05.250638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 10:20:05.250638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 10:20:05.250638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 10:20:05.250638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 10:20:05.250638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 10:20:05.250638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application conn.query(q) Aug 04 10:20:05.250638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 10:20:05.250638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 10:20:05.250638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 10:20:05.251615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application result.read() Aug 04 10:20:05.251615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 10:20:05.251615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 10:20:05.251615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 10:20:05.251615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 10:20:05.251615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 10:20:05.251615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 10:20:05.251615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 10:20:05.251615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 10:20:05.251615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 10:20:05.251615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:05.251615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 10:20:05.251615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:05.251615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:05.251615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 10:20:05.251615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 10:20:05.251615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 04 10:20:05.251615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 04 10:20:05.253786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 10:20:05.253786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application next(self.gen) Aug 04 10:20:05.253786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 10:20:05.253786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application yield resource Aug 04 10:20:05.253786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 10:20:05.253786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application next(self.gen) Aug 04 10:20:05.253786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 10:20:05.253786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 10:20:05.253786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 10:20:05.253786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 10:20:05.253786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 10:20:05.253786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise self.value Aug 04 10:20:05.253786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 10:20:05.253786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 10:20:05.253786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 10:20:05.253786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application session.commit() Aug 04 10:20:05.253786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 10:20:05.255101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 10:20:05.255101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 10:20:05.255101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 10:20:05.255101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 10:20:05.255101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self.session.flush() Aug 04 10:20:05.255101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 10:20:05.255101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 10:20:05.255101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 10:20:05.255101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 10:20:05.255101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 10:20:05.255101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application compat.raise_( Aug 04 10:20:05.255101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 10:20:05.255101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception Aug 04 10:20:05.255101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 10:20:05.255101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 10:20:05.255101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 10:20:05.255101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 10:20:05.257856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 10:20:05.257856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 10:20:05.257856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 10:20:05.257856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 10:20:05.257856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 04 10:20:05.257856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 04 10:20:05.257856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 10:20:05.257856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 10:20:05.257856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 10:20:05.257856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 10:20:05.257856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 10:20:05.257856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 10:20:05.257856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 10:20:05.257856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 10:20:05.257856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 10:20:05.258940 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 10:20:05.258940 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 10:20:05.258940 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception Aug 04 10:20:05.258940 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 10:20:05.258940 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 10:20:05.258940 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 10:20:05.258940 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 10:20:05.258940 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 10:20:05.258940 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 10:20:05.258940 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 10:20:05.258940 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application conn.query(q) Aug 04 10:20:05.258940 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 10:20:05.258940 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 10:20:05.258940 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 10:20:05.258940 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application result.read() Aug 04 10:20:05.258940 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 10:20:05.258940 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 10:20:05.260222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 10:20:05.260222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 10:20:05.260222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 10:20:05.260222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 10:20:05.260222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 10:20:05.260222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 10:20:05.260222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 10:20:05.260222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 04 10:20:05.260222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [parameters: {'id': 'df551692cecd4fd2944e68d1d64b6b34', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 04 10:20:05.260222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 10:20:05.260222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:05.260222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:20:05.260222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:05.260222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:05.260222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 04 10:20:05.261259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 04 10:20:05.261259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 10:20:05.261259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 10:20:05.261259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 04 10:20:05.261259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:05.261259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:20:05.261259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:05.261259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:05.261259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:05.261259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:05.261259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:05.261259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:05.261259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:05.261259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:05.261259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:05.261259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:05.261259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:05.262485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:05.262485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 04 10:20:05.262485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 04 10:20:05.262485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:05.262485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:05.262485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 04 10:20:05.262485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 04 10:20:05.262485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id df551692cecd4fd2944e68d1d64b6b34. Aug 04 10:20:05.262485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:05.262485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 481/940] 10.222.0.121 () {60 vars in 1145 bytes} [Thu Aug 4 10:20:05 2022] POST /identity/v3/projects => generated 262 bytes in 244 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:20:05.262485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b511957-d032-44bf-98a3-053ea515ee09 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:05.277021 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdea1d0-16d9-42cc-baf3-f9f3c3ed3ef0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:05.320336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a15a0-8171-44ab-8bd3-275801c86ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:05.321004 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a15a0-8171-44ab-8bd3-275801c86ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:05.321325 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a15a0-8171-44ab-8bd3-275801c86ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:05.321743 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a15a0-8171-44ab-8bd3-275801c86ec6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:05.332801 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a15a0-8171-44ab-8bd3-275801c86ec6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:05.334379 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a15a0-8171-44ab-8bd3-275801c86ec6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:05.348251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-b40a15a0-8171-44ab-8bd3-275801c86ec6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 10:20:05.348251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:05.348251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:05.348251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:05.348251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:05.348251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:05.348251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:05.348251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:05.348251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:05.348251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:05.348251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:05.348251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:05.348251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:20:05.348251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:20:05.348251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:20:05.348251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:20:05.350684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:05.350684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:05.350684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:20:05.350684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:20:05.350684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:20:05.350684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:20:05.350684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:05.350684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:05.350684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:05.350684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:05.350684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:05.350684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:05.350684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:05.350684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:05.350684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:05.350684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:05.350684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:05.350684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:05.353055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:05.353055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:05.353055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:20:05.353055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:20:05.353055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:20:05.353055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:20:05.353055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:20:05.353055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:20:05.353055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 10:20:05.353055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:05.359889 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 460/941] 10.222.0.121 () {56 vars in 1122 bytes} [Thu Aug 4 10:20:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:20:05.359889 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a15a0-8171-44ab-8bd3-275801c86ec6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:05.414186 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b511957-d032-44bf-98a3-053ea515ee09 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:05.445402 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:05.466926 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b49ba-ae6e-4890-bdbe-e61f77474b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:05.468192 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b49ba-ae6e-4890-bdbe-e61f77474b99 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:05.468648 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b49ba-ae6e-4890-bdbe-e61f77474b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:05.469068 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b49ba-ae6e-4890-bdbe-e61f77474b99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:05.477361 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a15a0-8171-44ab-8bd3-275801c86ec6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:05.481401 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b49ba-ae6e-4890-bdbe-e61f77474b99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:05.483568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b49ba-ae6e-4890-bdbe-e61f77474b99 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:05.500706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-f78b49ba-ae6e-4890-bdbe-e61f77474b99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:20:05.500706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:05.500706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:05.500706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:05.500706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:05.500706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:05.500706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:05.500706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:05.500706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:05.500706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:05.500706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:05.500706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:05.500706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:20:05.500706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:20:05.500706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:20:05.501945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:20:05.501945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:05.501945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:05.501945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:05.501945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:05.501945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:05.501945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:05.501945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:05.501945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:05.501945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:05.501945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:05.501945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:05.501945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:05.501945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:05.501945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:05.501945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:05.501945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:05.503107 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:20:05.503107 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:20:05.503107 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:20:05.503107 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:20:05.503107 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:20:05.503107 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:20:05.503107 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:20:05.503107 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:05.508005 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 482/942] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:20:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:05.513120 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b49ba-ae6e-4890-bdbe-e61f77474b99 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:05.531008 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4da81d-3a86-48e8-8b40-1dc1433563b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:05.532458 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4da81d-3a86-48e8-8b40-1dc1433563b4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:05.533193 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4da81d-3a86-48e8-8b40-1dc1433563b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:05.533823 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4da81d-3a86-48e8-8b40-1dc1433563b4 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:05.535268 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4da81d-3a86-48e8-8b40-1dc1433563b4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:05.536144 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4da81d-3a86-48e8-8b40-1dc1433563b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:05.538204 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4da81d-3a86-48e8-8b40-1dc1433563b4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:05.574700 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 461/943] 10.222.0.121 () {56 vars in 1154 bytes} [Thu Aug 4 10:20:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:05.576531 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4da81d-3a86-48e8-8b40-1dc1433563b4 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:05.595851 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:05.663010 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b49ba-ae6e-4890-bdbe-e61f77474b99 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:05.713712 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2213b26-2ed1-463d-9097-de6573570051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:05.714505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2213b26-2ed1-463d-9097-de6573570051 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:05.714870 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2213b26-2ed1-463d-9097-de6573570051 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:05.715228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2213b26-2ed1-463d-9097-de6573570051 None admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:05.716180 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2213b26-2ed1-463d-9097-de6573570051 None admin] RBAC: Adding query filter params (name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:05.716577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2213b26-2ed1-463d-9097-de6573570051 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:05.719005 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2213b26-2ed1-463d-9097-de6573570051 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:05.733040 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4da81d-3a86-48e8-8b40-1dc1433563b4 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:05.752520 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 483/944] 10.222.0.121 () {56 vars in 1117 bytes} [Thu Aug 4 10:20:05 2022] GET /identity/v3/projects?name=service => generated 413 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:05.786395 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63550d-b78b-43c4-be2f-1a99666c9eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:05.787540 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63550d-b78b-43c4-be2f-1a99666c9eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:05.788122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63550d-b78b-43c4-be2f-1a99666c9eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:05.788588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63550d-b78b-43c4-be2f-1a99666c9eb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:05.801435 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63550d-b78b-43c4-be2f-1a99666c9eb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:05.803408 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63550d-b78b-43c4-be2f-1a99666c9eb0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:05.821904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-cf63550d-b78b-43c4-be2f-1a99666c9eb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:20:05.821904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:05.821904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:05.821904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:05.821904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:05.821904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:05.821904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:05.821904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:05.821904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:05.821904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:05.821904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:05.821904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:05.821904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:20:05.821904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:20:05.821904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:20:05.822988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:20:05.822988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:05.822988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:05.822988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:05.822988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:05.822988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:05.822988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:05.822988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:05.822988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:05.822988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:05.822988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:05.822988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:05.822988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:05.822988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:05.822988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:05.822988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:05.822988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:05.823968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:20:05.823968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:20:05.823968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:20:05.823968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:05.831126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 462/945] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:20:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:20:05.833059 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2213b26-2ed1-463d-9097-de6573570051 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:05.888733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:05.978314 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2213b26-2ed1-463d-9097-de6573570051 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:06.027712 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66121aad-c62b-4d33-bb08-b4ed3a8cceb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:06.028564 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66121aad-c62b-4d33-bb08-b4ed3a8cceb5 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:06.028955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66121aad-c62b-4d33-bb08-b4ed3a8cceb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:06.029363 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66121aad-c62b-4d33-bb08-b4ed3a8cceb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:06.030204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66121aad-c62b-4d33-bb08-b4ed3a8cceb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:06.030626 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66121aad-c62b-4d33-bb08-b4ed3a8cceb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:06.032189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66121aad-c62b-4d33-bb08-b4ed3a8cceb5 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:06.049341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 484/946] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:20:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:06.055513 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63550d-b78b-43c4-be2f-1a99666c9eb0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:06.144544 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:06.202361 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63550d-b78b-43c4-be2f-1a99666c9eb0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:06.242629 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b187b-513b-47c7-8623-3b2d57b8d552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:06.243330 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b187b-513b-47c7-8623-3b2d57b8d552 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:06.243615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b187b-513b-47c7-8623-3b2d57b8d552 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:06.243730 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b187b-513b-47c7-8623-3b2d57b8d552 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:06.251541 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b187b-513b-47c7-8623-3b2d57b8d552 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:06.253053 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b187b-513b-47c7-8623-3b2d57b8d552 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:06.265488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-036b187b-513b-47c7-8623-3b2d57b8d552 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:20:06.265488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:06.265488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:06.265488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:06.265488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:06.265488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:06.265488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:06.265488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:06.265488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:06.265488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:06.265488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:06.265488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:06.265488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:20:06.265488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:20:06.265488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:20:06.266536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:20:06.266536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:06.266536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:06.266536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:06.266536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:06.266536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:06.266536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:06.266536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:06.266536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:06.266536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:06.266536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:06.266536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:06.266536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:06.266536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:06.266536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:06.266536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:06.266536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:06.268114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:20:06.268114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:20:06.268114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:20:06.268114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:20:06.268114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:20:06.268114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:20:06.268114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:20:06.268114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:06.268114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 463/947] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:20:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:06.275555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66121aad-c62b-4d33-bb08-b4ed3a8cceb5 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:06.449428 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:06.458011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66121aad-c62b-4d33-bb08-b4ed3a8cceb5 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:06.514718 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d053f49-77ce-47e8-9e17-b35cf3b3f42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:06.515774 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d053f49-77ce-47e8-9e17-b35cf3b3f42a None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:06.516332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d053f49-77ce-47e8-9e17-b35cf3b3f42a None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:06.516809 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d053f49-77ce-47e8-9e17-b35cf3b3f42a None admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:06.517797 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d053f49-77ce-47e8-9e17-b35cf3b3f42a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:06.518337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d053f49-77ce-47e8-9e17-b35cf3b3f42a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:06.520043 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d053f49-77ce-47e8-9e17-b35cf3b3f42a None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:06.539872 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 485/948] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:20:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:06.541730 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b187b-513b-47c7-8623-3b2d57b8d552 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:06.725510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b187b-513b-47c7-8623-3b2d57b8d552 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:06.746819 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:06.768701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f77f8-4b8b-4ed3-a419-ec2af882ed7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:06.769347 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f77f8-4b8b-4ed3-a419-ec2af882ed7a None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:06.769637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f77f8-4b8b-4ed3-a419-ec2af882ed7a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:06.769919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f77f8-4b8b-4ed3-a419-ec2af882ed7a None admin] PATH_INFO: `/v3/projects/22768aba760b4307aab131d1312b2bf6/users/efe6e59228bd49bfbff25a11eb682426/roles/9395de2ca6f34dfc85ea530fc0a14f45` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:06.772411 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f77f8-4b8b-4ed3-a419-ec2af882ed7a None admin] RBAC: Authorizing `identity:create_grant(project_id=22768aba760b4307aab131d1312b2bf6, user_id=efe6e59228bd49bfbff25a11eb682426, role_id=9395de2ca6f34dfc85ea530fc0a14f45)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:06.773948 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f77f8-4b8b-4ed3-a419-ec2af882ed7a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:06.793037 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 464/949] 10.222.0.121 () {58 vars in 1444 bytes} [Thu Aug 4 10:20:06 2022] PUT /identity/v3/projects/22768aba760b4307aab131d1312b2bf6/users/efe6e59228bd49bfbff25a11eb682426/roles/9395de2ca6f34dfc85ea530fc0a14f45 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:20:06.813122 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed023c17-1405-4c08-8243-d5241940cf31 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:06.813492 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed023c17-1405-4c08-8243-d5241940cf31 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:06.813690 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed023c17-1405-4c08-8243-d5241940cf31 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:06.815577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 486/950] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:06 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:06.821780 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a7264-a704-4f3e-8fac-e1c43f86e6fe None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:06.822125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a7264-a704-4f3e-8fac-e1c43f86e6fe None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:06.822339 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a7264-a704-4f3e-8fac-e1c43f86e6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:06.843845 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-fc1a7264-a704-4f3e-8fac-e1c43f86e6fe None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:06.845234 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fc1a7264-a704-4f3e-8fac-e1c43f86e6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:06.935050 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 465/951] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:06 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:06.961098 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed023c17-1405-4c08-8243-d5241940cf31 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:07.094101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed023c17-1405-4c08-8243-d5241940cf31 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:07.138529 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b8bb0-e2ad-4d39-954e-67fe63a97edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:07.139455 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b8bb0-e2ad-4d39-954e-67fe63a97edc None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:07.139774 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b8bb0-e2ad-4d39-954e-67fe63a97edc None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:07.140086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b8bb0-e2ad-4d39-954e-67fe63a97edc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:07.140750 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b8bb0-e2ad-4d39-954e-67fe63a97edc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:07.142257 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b8bb0-e2ad-4d39-954e-67fe63a97edc None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:07.153319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-3d6b8bb0-e2ad-4d39-954e-67fe63a97edc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:20:07.153319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:07.153319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:20:07.153319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:20:07.153319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:20:07.153319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:20:07.153319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:20:07.153319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:20:07.153319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:20:07.153319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:07.153319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:20:07.153319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:07.153319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:07.153319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:07.153319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:07.153319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:07.153319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:07.155708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:07.155708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:07.155708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:07.155708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:07.155708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:07.155708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:07.155708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:20:07.155708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:20:07.155708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:20:07.155708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:20:07.155708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:07.155708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:07.155708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:07.155708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:07.155708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:07.155708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:07.157350 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:07.157350 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:07.157350 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:07.157350 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:07.157350 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:07.157350 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:07.157350 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:07.157350 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:07.157350 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:07.157350 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:07.157350 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:20:07.157350 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:20:07.157350 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:20:07.157350 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:07.157350 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 487/952] 10.222.0.121 () {56 vars in 1131 bytes} [Thu Aug 4 10:20:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:07.161594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a7264-a704-4f3e-8fac-e1c43f86e6fe None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:07.178344 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417ffb8-cf2a-4a3b-a257-5b970c85e2a4 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:07.178741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417ffb8-cf2a-4a3b-a257-5b970c85e2a4 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:07.179508 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417ffb8-cf2a-4a3b-a257-5b970c85e2a4 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:07.181479 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 488/953] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:07 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:07.187506 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a80e5-159e-4f80-a9a6-35611d15c56e None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:07.187833 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a80e5-159e-4f80-a9a6-35611d15c56e None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:07.188109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a80e5-159e-4f80-a9a6-35611d15c56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:07.213479 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-7f8a80e5-159e-4f80-a9a6-35611d15c56e None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:07.216757 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-7f8a80e5-159e-4f80-a9a6-35611d15c56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:07.252413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:07.314653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a7264-a704-4f3e-8fac-e1c43f86e6fe None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:07.323355 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 489/954] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:07 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:07.332902 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:07.346093 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a80e5-159e-4f80-a9a6-35611d15c56e None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:07.367239 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac7645-5933-4336-b42b-a7c8e4808755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:07.368259 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac7645-5933-4336-b42b-a7c8e4808755 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:07.368721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac7645-5933-4336-b42b-a7c8e4808755 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:07.369159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac7645-5933-4336-b42b-a7c8e4808755 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:07.370014 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac7645-5933-4336-b42b-a7c8e4808755 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:07.370492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac7645-5933-4336-b42b-a7c8e4808755 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:07.372116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac7645-5933-4336-b42b-a7c8e4808755 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:07.378324 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 466/955] 10.222.0.121 () {56 vars in 1127 bytes} [Thu Aug 4 10:20:07 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:07.383808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac7645-5933-4336-b42b-a7c8e4808755 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:07.499214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a80e5-159e-4f80-a9a6-35611d15c56e None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:07.542324 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac7645-5933-4336-b42b-a7c8e4808755 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:07.544735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603bdf9-e182-47ac-abee-b7cb51527a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:07.545651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603bdf9-e182-47ac-abee-b7cb51527a33 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:07.546228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603bdf9-e182-47ac-abee-b7cb51527a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:07.546729 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603bdf9-e182-47ac-abee-b7cb51527a33 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:07.547604 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603bdf9-e182-47ac-abee-b7cb51527a33 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:07.549591 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603bdf9-e182-47ac-abee-b7cb51527a33 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:07.565134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-b603bdf9-e182-47ac-abee-b7cb51527a33 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 10:20:07.565134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:07.565134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:20:07.565134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:20:07.565134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:20:07.565134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:20:07.565134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:20:07.565134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:20:07.565134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 10:20:07.565134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:07.565134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:20:07.565134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:07.565134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:07.565134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:07.565134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:07.565134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:07.565134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:07.567505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:07.567505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:07.567505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:07.567505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:07.567505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:07.567505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:07.567505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:20:07.567505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:20:07.567505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:20:07.567505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:20:07.567505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:07.567505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:07.567505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:07.567505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:07.567505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:07.567505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:07.569417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:07.569417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:07.569417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:07.569417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:07.569417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:07.569417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:07.569417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:07.569417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:07.569417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:07.569417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:07.569417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:20:07.569417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:20:07.569417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 10:20:07.569417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:07.574770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 490/956] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:20:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:20:07.586082 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603bdf9-e182-47ac-abee-b7cb51527a33 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:07.589217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5aabb-8fd5-455d-87d8-2c9319966742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:07.590057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5aabb-8fd5-455d-87d8-2c9319966742 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:07.590495 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5aabb-8fd5-455d-87d8-2c9319966742 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:07.590936 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5aabb-8fd5-455d-87d8-2c9319966742 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:07.591802 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5aabb-8fd5-455d-87d8-2c9319966742 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:07.592159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5aabb-8fd5-455d-87d8-2c9319966742 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:07.593518 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5aabb-8fd5-455d-87d8-2c9319966742 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:07.601572 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 467/957] 10.222.0.121 () {56 vars in 1127 bytes} [Thu Aug 4 10:20:07 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:07.609178 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5aabb-8fd5-455d-87d8-2c9319966742 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:07.622835 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:07.657744 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:07.736544 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5aabb-8fd5-455d-87d8-2c9319966742 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:07.737202 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603bdf9-e182-47ac-abee-b7cb51527a33 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:07.778862 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1687573-c150-4201-b4dc-53cd5db9d2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:07.780154 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1687573-c150-4201-b4dc-53cd5db9d2bb None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:07.781020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1687573-c150-4201-b4dc-53cd5db9d2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:07.781656 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1687573-c150-4201-b4dc-53cd5db9d2bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:07.782676 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1687573-c150-4201-b4dc-53cd5db9d2bb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:07.784800 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1687573-c150-4201-b4dc-53cd5db9d2bb None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:07.786723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2c518-7293-4725-b4ec-fb41192dc65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:07.787437 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2c518-7293-4725-b4ec-fb41192dc65b None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:07.787784 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2c518-7293-4725-b4ec-fb41192dc65b None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:07.788160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2c518-7293-4725-b4ec-fb41192dc65b None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:07.788934 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2c518-7293-4725-b4ec-fb41192dc65b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:07.789357 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2c518-7293-4725-b4ec-fb41192dc65b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:07.790784 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2c518-7293-4725-b4ec-fb41192dc65b None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:07.799999 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 491/958] 10.222.0.121 () {56 vars in 1117 bytes} [Thu Aug 4 10:20:07 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:07.805039 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2c518-7293-4725-b4ec-fb41192dc65b None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:07.823339 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 468/959] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:20:07 2022] POST /identity/v3/endpoints => generated 351 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:20:07.941533 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:07.969421 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2c518-7293-4725-b4ec-fb41192dc65b None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:08.025006 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736b87c-6799-4fd1-a140-2549af1e1874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:08.025659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736b87c-6799-4fd1-a140-2549af1e1874 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:08.025939 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736b87c-6799-4fd1-a140-2549af1e1874 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:08.026211 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736b87c-6799-4fd1-a140-2549af1e1874 None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:08.026911 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b736b87c-6799-4fd1-a140-2549af1e1874 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:08.027213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b736b87c-6799-4fd1-a140-2549af1e1874 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:08.029164 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b736b87c-6799-4fd1-a140-2549af1e1874 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:08.036618 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 492/960] 10.222.0.121 () {56 vars in 1117 bytes} [Thu Aug 4 10:20:07 2022] GET /identity/v3/services?type=compute => generated 125 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:08.172952 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c0429-3110-43cd-b4b4-1a9fe1b5ab83 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:08.173279 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c0429-3110-43cd-b4b4-1a9fe1b5ab83 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:08.173506 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c0429-3110-43cd-b4b4-1a9fe1b5ab83 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:08.175276 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 469/961] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:08 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:08.181643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64243e49-cdba-4255-ae1b-9b58efdcc30a None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:08.182172 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64243e49-cdba-4255-ae1b-9b58efdcc30a None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:08.182553 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64243e49-cdba-4255-ae1b-9b58efdcc30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:08.209376 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-64243e49-cdba-4255-ae1b-9b58efdcc30a None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:08.211892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-64243e49-cdba-4255-ae1b-9b58efdcc30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:08.335986 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 493/962] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:08 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:08.360990 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c0429-3110-43cd-b4b4-1a9fe1b5ab83 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:08.524173 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:08.536887 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c0429-3110-43cd-b4b4-1a9fe1b5ab83 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:08.578199 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbc725-1be7-405c-9f64-1df35a8a5755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:08.578838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbc725-1be7-405c-9f64-1df35a8a5755 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:08.579123 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbc725-1be7-405c-9f64-1df35a8a5755 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:08.579418 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbc725-1be7-405c-9f64-1df35a8a5755 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:08.580143 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbc725-1be7-405c-9f64-1df35a8a5755 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:08.581941 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbc725-1be7-405c-9f64-1df35a8a5755 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:08.585736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: WARNING keystone.server.flask.common [None req-70fbc725-1be7-405c-9f64-1df35a8a5755 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 04 10:20:08.598969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.sql.core [None req-70fbc725-1be7-405c-9f64-1df35a8a5755 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 10:20:08.598969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 04 10:20:08.598969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [parameters: {'user_id': '5bdff12d45f04ea1a45b3fa5435353a5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 04 10:20:08.598969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80236) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 10:20:08.617530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-70fbc725-1be7-405c-9f64-1df35a8a5755 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 04 10:20:08.617530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:08.617530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 10:20:08.617530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 10:20:08.617530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 10:20:08.617530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 10:20:08.617530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 10:20:08.617530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 10:20:08.617530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 10:20:08.617530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application conn.query(q) Aug 04 10:20:08.617530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 10:20:08.617530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 10:20:08.617530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 10:20:08.617530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application result.read() Aug 04 10:20:08.617530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 10:20:08.619272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 10:20:08.619272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 10:20:08.619272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 10:20:08.619272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 10:20:08.619272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 10:20:08.619272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 10:20:08.619272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 10:20:08.619272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 10:20:08.619272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:08.619272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 10:20:08.619272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:08.619272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:08.619272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 10:20:08.619272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 10:20:08.619272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 04 10:20:08.619272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 04 10:20:08.619272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 10:20:08.619272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application next(self.gen) Aug 04 10:20:08.621220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 10:20:08.621220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application yield resource Aug 04 10:20:08.621220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 10:20:08.621220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application next(self.gen) Aug 04 10:20:08.621220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 10:20:08.621220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 10:20:08.621220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 10:20:08.621220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 10:20:08.621220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 10:20:08.621220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise self.value Aug 04 10:20:08.621220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 10:20:08.621220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 10:20:08.621220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 10:20:08.621220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application session.commit() Aug 04 10:20:08.621220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 10:20:08.621220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 10:20:08.621220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 10:20:08.623033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 10:20:08.623033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 10:20:08.623033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self.session.flush() Aug 04 10:20:08.623033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 10:20:08.623033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 10:20:08.623033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 10:20:08.623033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 10:20:08.623033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 10:20:08.623033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application compat.raise_( Aug 04 10:20:08.623033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 10:20:08.623033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception Aug 04 10:20:08.623033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 10:20:08.623033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 10:20:08.623033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 10:20:08.623033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 10:20:08.623033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 10:20:08.623033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 10:20:08.626566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 10:20:08.626566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 10:20:08.626566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 04 10:20:08.626566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 04 10:20:08.626566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 10:20:08.626566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 10:20:08.626566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 10:20:08.626566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 10:20:08.626566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 10:20:08.626566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 10:20:08.626566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 10:20:08.626566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 10:20:08.626566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 10:20:08.626566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 10:20:08.626566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 10:20:08.626566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception Aug 04 10:20:08.633231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 10:20:08.633231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 10:20:08.633231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 10:20:08.633231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 10:20:08.633231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 10:20:08.633231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 10:20:08.633231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 10:20:08.633231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application conn.query(q) Aug 04 10:20:08.633231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 10:20:08.633231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 10:20:08.633231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 10:20:08.633231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application result.read() Aug 04 10:20:08.633231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 10:20:08.633231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 10:20:08.633231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 10:20:08.633231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 10:20:08.636447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 10:20:08.636447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 10:20:08.636447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 10:20:08.636447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 10:20:08.636447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 10:20:08.636447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 04 10:20:08.636447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [parameters: {'user_id': '5bdff12d45f04ea1a45b3fa5435353a5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 04 10:20:08.636447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 10:20:08.636447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:08.636447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:20:08.636447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:08.636447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:08.636447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:08.636447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:08.636447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:08.639567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:08.639567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:08.639567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:08.639567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:08.639567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:08.639567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:08.639567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:08.639567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 04 10:20:08.639567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 04 10:20:08.639567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:08.639567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:08.639567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:20:08.639567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:20:08.639567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:20:08.639567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:20:08.639567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 04 10:20:08.639567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 04 10:20:08.644386 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 04 10:20:08.644386 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 04 10:20:08.644386 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 10:20:08.644386 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 10:20:08.644386 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 04 10:20:08.644386 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:08.644386 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 470/963] 10.222.0.121 () {60 vars in 1136 bytes} [Thu Aug 4 10:20:08 2022] POST /identity/v3/users => generated 160 bytes in 259 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:08.645923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64243e49-cdba-4255-ae1b-9b58efdcc30a None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:08.793560 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:08.839424 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64243e49-cdba-4255-ae1b-9b58efdcc30a None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:08.886473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78750c8-02ca-4347-946d-813f8e148612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:08.887154 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78750c8-02ca-4347-946d-813f8e148612 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:08.887436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78750c8-02ca-4347-946d-813f8e148612 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:08.887713 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78750c8-02ca-4347-946d-813f8e148612 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:08.898022 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78750c8-02ca-4347-946d-813f8e148612 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:08.899528 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78750c8-02ca-4347-946d-813f8e148612 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:08.913375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-e78750c8-02ca-4347-946d-813f8e148612 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 10:20:08.913375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:08.913375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:08.913375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:08.913375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:08.913375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:08.913375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:08.913375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:08.913375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:08.913375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:08.913375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:08.913375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:08.913375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:20:08.913375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:20:08.913375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:20:08.913375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:20:08.914542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:08.914542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:08.914542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:20:08.914542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:20:08.914542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:20:08.914542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:20:08.914542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:08.914542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:08.914542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:08.914542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:08.914542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:08.914542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:08.914542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:08.914542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:08.914542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:08.914542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:08.914542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:08.914542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:08.915549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:08.915549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:08.915549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:20:08.915549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:20:08.915549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:20:08.915549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:20:08.915549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:20:08.915549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:20:08.915549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 10:20:08.915549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:08.915549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 494/964] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:20:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:08.921095 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbc725-1be7-405c-9f64-1df35a8a5755 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:08.975597 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49722c-98a9-415e-b935-b5af11e04ae0 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:08.976012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49722c-98a9-415e-b935-b5af11e04ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:08.976207 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49722c-98a9-415e-b935-b5af11e04ae0 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:08.981889 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 495/965] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:08 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:08.983755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e422771-1482-49c4-bd42-1b61c652d7d8 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:08.984080 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e422771-1482-49c4-bd42-1b61c652d7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:08.984363 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e422771-1482-49c4-bd42-1b61c652d7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:09.003820 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-3e422771-1482-49c4-bd42-1b61c652d7d8 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:09.005116 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-3e422771-1482-49c4-bd42-1b61c652d7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:09.075260 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbc725-1be7-405c-9f64-1df35a8a5755 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:09.099286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 496/966] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:08 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:09.116196 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e422771-1482-49c4-bd42-1b61c652d7d8 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:09.118855 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9d2ac-4e49-49d9-97e5-769ca4d30fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:09.120355 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9d2ac-4e49-49d9-97e5-769ca4d30fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:09.120639 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9d2ac-4e49-49d9-97e5-769ca4d30fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:09.120915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9d2ac-4e49-49d9-97e5-769ca4d30fb5 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:09.121730 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b9d2ac-4e49-49d9-97e5-769ca4d30fb5 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:09.122038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b9d2ac-4e49-49d9-97e5-769ca4d30fb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:09.123831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b9d2ac-4e49-49d9-97e5-769ca4d30fb5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:09.196412 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 471/967] 10.222.0.121 () {56 vars in 1106 bytes} [Thu Aug 4 10:20:08 2022] GET /identity/v3/users?name=cinder => generated 356 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:09.223275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:09.279553 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e422771-1482-49c4-bd42-1b61c652d7d8 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:09.328651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da23ed-d3bd-4d57-8241-3d4fec58c586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:09.329783 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da23ed-d3bd-4d57-8241-3d4fec58c586 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:09.330305 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da23ed-d3bd-4d57-8241-3d4fec58c586 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:09.330835 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da23ed-d3bd-4d57-8241-3d4fec58c586 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:09.348452 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da23ed-d3bd-4d57-8241-3d4fec58c586 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:20:09.355718 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-74da23ed-d3bd-4d57-8241-3d4fec58c586 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 10:20:09.355718 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:09.355718 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:09.355718 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:09.355718 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:09.355718 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:09.355718 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:09.355718 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:09.355718 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:09.355718 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:09.355718 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:09.355718 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:09.355718 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:20:09.355718 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:20:09.355718 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:20:09.355718 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:20:09.356892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:20:09.356892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:20:09.356892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:20:09.356892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:20:09.356892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:09.356892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:09.356892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:09.356892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:09.356892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:09.356892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:09.356892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:09.356892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:09.356892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:09.356892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:09.356892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:09.356892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:09.357776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:09.357776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:09.357776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:09.357776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:09.357776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:20:09.357776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:20:09.357776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:20:09.357776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:20:09.357776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:20:09.357776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:20:09.357776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 10:20:09.357776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:09.361236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 497/968] 10.222.0.121 () {56 vars in 1125 bytes} [Thu Aug 4 10:20:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:09.368122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9d2ac-4e49-49d9-97e5-769ca4d30fb5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:09.389214 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:09.504107 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9d2ac-4e49-49d9-97e5-769ca4d30fb5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:09.553521 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcc056-ec9f-4c76-92b5-4b3034a6a010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:09.554580 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcc056-ec9f-4c76-92b5-4b3034a6a010 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:09.554965 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcc056-ec9f-4c76-92b5-4b3034a6a010 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:09.555340 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcc056-ec9f-4c76-92b5-4b3034a6a010 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:09.556405 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbcc056-ec9f-4c76-92b5-4b3034a6a010 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:09.557125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbcc056-ec9f-4c76-92b5-4b3034a6a010 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:09.559038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbcc056-ec9f-4c76-92b5-4b3034a6a010 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:09.574390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 472/969] 10.222.0.121 () {56 vars in 1120 bytes} [Thu Aug 4 10:20:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:09.581760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da23ed-d3bd-4d57-8241-3d4fec58c586 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:09.724746 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da23ed-d3bd-4d57-8241-3d4fec58c586 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:09.750614 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:09.765809 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c146c5a-497e-4a08-9357-6f06e03534a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:09.766769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c146c5a-497e-4a08-9357-6f06e03534a9 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:09.767153 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c146c5a-497e-4a08-9357-6f06e03534a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:09.767535 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c146c5a-497e-4a08-9357-6f06e03534a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:09.779271 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c146c5a-497e-4a08-9357-6f06e03534a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:09.781021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c146c5a-497e-4a08-9357-6f06e03534a9 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:09.798778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-5c146c5a-497e-4a08-9357-6f06e03534a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:20:09.798778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:09.798778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:09.798778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:09.798778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:09.798778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:09.798778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:09.798778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:09.798778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:09.798778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:09.798778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:09.798778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:09.798778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:20:09.798778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:20:09.798778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:20:09.800670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:20:09.800670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:09.800670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:09.800670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:09.800670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:09.800670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:09.800670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:09.800670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:09.800670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:09.800670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:09.800670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:09.800670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:09.800670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:09.800670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:09.800670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:09.800670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:09.800670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:09.802651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:20:09.802651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:20:09.802651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:20:09.802651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:09.806804 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 498/970] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:20:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:09.808309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcc056-ec9f-4c76-92b5-4b3034a6a010 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:09.920372 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:09.948793 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcc056-ec9f-4c76-92b5-4b3034a6a010 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:09.988975 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ea841-2315-477b-ab36-90d4b49af478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:09.989551 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ea841-2315-477b-ab36-90d4b49af478 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:09.990420 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ea841-2315-477b-ab36-90d4b49af478 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:09.990420 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ea841-2315-477b-ab36-90d4b49af478 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:09.990906 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ea841-2315-477b-ab36-90d4b49af478 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:09.991222 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ea841-2315-477b-ab36-90d4b49af478 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:09.992658 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ea841-2315-477b-ab36-90d4b49af478 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:10.006510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 473/971] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:20:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:10.012219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c146c5a-497e-4a08-9357-6f06e03534a9 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:10.148914 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c146c5a-497e-4a08-9357-6f06e03534a9 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:10.195428 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e427c-2248-4ca4-8f7e-38d9956246e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:10.196004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e427c-2248-4ca4-8f7e-38d9956246e5 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:10.196341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e427c-2248-4ca4-8f7e-38d9956246e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:10.196619 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e427c-2248-4ca4-8f7e-38d9956246e5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:10.196771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f4cfd-e87c-41c8-bdb9-77ea7c595869 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:10.196940 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f4cfd-e87c-41c8-bdb9-77ea7c595869 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:10.197196 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f4cfd-e87c-41c8-bdb9-77ea7c595869 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:10.198862 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 474/972] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:10 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 10:20:10.204703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd4238-fe87-4c2a-818e-14b566ad35f4 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:10.204972 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd4238-fe87-4c2a-818e-14b566ad35f4 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:10.205253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd4238-fe87-4c2a-818e-14b566ad35f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:10.206511 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e427c-2248-4ca4-8f7e-38d9956246e5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:10.208066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e427c-2248-4ca4-8f7e-38d9956246e5 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:10.222112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-4f8e427c-2248-4ca4-8f7e-38d9956246e5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 10:20:10.222112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:10.222112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:10.222112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:10.222112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:10.222112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:10.222112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:10.222112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:10.222112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:10.222112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:10.222112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:10.222112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:10.222112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:20:10.222112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:20:10.222112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:20:10.222112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:20:10.223301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:10.223301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:10.223301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:20:10.223301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:20:10.223301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:20:10.223301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:20:10.223301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:10.223301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:10.223301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:10.223301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:10.223301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:10.223301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:10.223301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:10.223301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:10.223301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:10.223301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:10.223301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:10.223301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:10.224413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:10.224413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:10.224413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:20:10.224413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:20:10.224413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:20:10.224413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:20:10.224413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:20:10.224413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:20:10.224413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 10:20:10.224413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:10.224413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 499/973] 10.222.0.121 () {56 vars in 1122 bytes} [Thu Aug 4 10:20:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:10.229119 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-bcfd4238-fe87-4c2a-818e-14b566ad35f4 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:10.229311 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0cc655-7594-42a9-8f28-e504e2c49728 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:10.229311 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0cc655-7594-42a9-8f28-e504e2c49728 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:10.229503 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bcfd4238-fe87-4c2a-818e-14b566ad35f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:10.229586 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0cc655-7594-42a9-8f28-e504e2c49728 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:10.229586 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 500/974] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:10.230085 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0cc655-7594-42a9-8f28-e504e2c49728 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:10.273827 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:10.315829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 475/975] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:10 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:20:10.317149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783d23a-ccf7-4970-9c1d-9a1f4da44abb None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:10.317473 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783d23a-ccf7-4970-9c1d-9a1f4da44abb None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:10.317756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783d23a-ccf7-4970-9c1d-9a1f4da44abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:10.338730 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8783d23a-ccf7-4970-9c1d-9a1f4da44abb None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:10.339995 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8783d23a-ccf7-4970-9c1d-9a1f4da44abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:10.358977 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0cc655-7594-42a9-8f28-e504e2c49728 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:10.406381 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a647446-8735-41d8-b98f-53384bdeabb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:10.407021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a647446-8735-41d8-b98f-53384bdeabb1 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:10.407305 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a647446-8735-41d8-b98f-53384bdeabb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:10.407580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a647446-8735-41d8-b98f-53384bdeabb1 None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:10.408467 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a647446-8735-41d8-b98f-53384bdeabb1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:10.408772 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a647446-8735-41d8-b98f-53384bdeabb1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:10.410238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a647446-8735-41d8-b98f-53384bdeabb1 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:10.413139 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:10.426160 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 476/976] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:10 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:10.427471 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8783d23a-ccf7-4970-9c1d-9a1f4da44abb None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:10.450739 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 501/977] 10.222.0.121 () {56 vars in 1154 bytes} [Thu Aug 4 10:20:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:10.451626 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a647446-8735-41d8-b98f-53384bdeabb1 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:10.553162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:10.563110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8783d23a-ccf7-4970-9c1d-9a1f4da44abb None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:10.564716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a647446-8735-41d8-b98f-53384bdeabb1 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:10.602343 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99359134-a49e-4cb6-881c-eecbde8d6855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:10.603015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99359134-a49e-4cb6-881c-eecbde8d6855 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:10.603303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99359134-a49e-4cb6-881c-eecbde8d6855 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:10.603586 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99359134-a49e-4cb6-881c-eecbde8d6855 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:10.604273 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99359134-a49e-4cb6-881c-eecbde8d6855 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:10.605688 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99359134-a49e-4cb6-881c-eecbde8d6855 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:10.606299 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be09046-37c8-4806-909e-a559bda296cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:10.606940 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be09046-37c8-4806-909e-a559bda296cf None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:10.607222 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be09046-37c8-4806-909e-a559bda296cf None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:10.607724 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be09046-37c8-4806-909e-a559bda296cf None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:10.608554 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be09046-37c8-4806-909e-a559bda296cf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:10.611108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be09046-37c8-4806-909e-a559bda296cf None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:10.616923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-99359134-a49e-4cb6-881c-eecbde8d6855 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:20:10.616923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:10.616923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:20:10.616923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:20:10.616923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:20:10.616923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:20:10.616923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:20:10.616923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:20:10.616923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:20:10.616923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:10.616923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:20:10.616923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:10.616923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:10.616923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:10.616923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:10.616923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:10.616923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:10.618108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:10.618108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:10.618108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:10.618108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:10.618108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:10.618108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:10.618108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:20:10.618108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:20:10.618108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:20:10.618108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:20:10.618108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:10.618108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:10.618108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:10.618108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:10.618108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:10.618108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:10.619048 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:10.619048 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:10.619048 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:10.619048 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:10.619048 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:10.619048 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:10.619048 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:10.619048 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:10.619048 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:10.619048 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:10.619048 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:20:10.619048 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:20:10.619048 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:20:10.619048 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:10.619048 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 502/978] 10.222.0.121 () {56 vars in 1131 bytes} [Thu Aug 4 10:20:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:10.619893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99359134-a49e-4cb6-881c-eecbde8d6855 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:10.636522 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 477/979] 10.222.0.121 () {60 vars in 1146 bytes} [Thu Aug 4 10:20:10 2022] POST /identity/v3/services => generated 243 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:20:10.637888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be09046-37c8-4806-909e-a559bda296cf None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:10.738948 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:10.785757 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99359134-a49e-4cb6-881c-eecbde8d6855 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:10.808054 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be09046-37c8-4806-909e-a559bda296cf None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:10.844477 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8d7b3-c9f5-49f2-bac8-b392bc825df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:10.845129 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8d7b3-c9f5-49f2-bac8-b392bc825df7 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:10.845412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8d7b3-c9f5-49f2-bac8-b392bc825df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:10.845689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8d7b3-c9f5-49f2-bac8-b392bc825df7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:10.854033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9bc05-ffac-4c2f-a0fa-647b76cce1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:10.854705 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9bc05-ffac-4c2f-a0fa-647b76cce1d6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:10.854998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9bc05-ffac-4c2f-a0fa-647b76cce1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:10.855277 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9bc05-ffac-4c2f-a0fa-647b76cce1d6 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:10.856091 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9bc05-ffac-4c2f-a0fa-647b76cce1d6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:10.856408 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9bc05-ffac-4c2f-a0fa-647b76cce1d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:10.857861 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9bc05-ffac-4c2f-a0fa-647b76cce1d6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:10.867983 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 478/980] 10.222.0.121 () {56 vars in 1127 bytes} [Thu Aug 4 10:20:10 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:10.874242 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9bc05-ffac-4c2f-a0fa-647b76cce1d6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:10.881230 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:10.932701 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8d7b3-c9f5-49f2-bac8-b392bc825df7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:10.934166 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8d7b3-c9f5-49f2-bac8-b392bc825df7 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:10.989556 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9bc05-ffac-4c2f-a0fa-647b76cce1d6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:11.004443 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:11.027488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c94a70-4b2e-4719-b3fa-bb1909e843dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:11.028143 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c94a70-4b2e-4719-b3fa-bb1909e843dd None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:11.028430 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c94a70-4b2e-4719-b3fa-bb1909e843dd None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:11.028707 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c94a70-4b2e-4719-b3fa-bb1909e843dd None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:11.029399 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c94a70-4b2e-4719-b3fa-bb1909e843dd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:11.029709 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c94a70-4b2e-4719-b3fa-bb1909e843dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:11.031127 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c94a70-4b2e-4719-b3fa-bb1909e843dd None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:11.036625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 479/981] 10.222.0.121 () {56 vars in 1127 bytes} [Thu Aug 4 10:20:10 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:11.042437 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c94a70-4b2e-4719-b3fa-bb1909e843dd None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:11.049000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 503/982] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:20:10 2022] GET /identity/v3/auth/tokens => generated 1572 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:20:11.050059 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8d7b3-c9f5-49f2-bac8-b392bc825df7 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:11.128821 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:11.173730 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c94a70-4b2e-4719-b3fa-bb1909e843dd None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:11.192955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8d7b3-c9f5-49f2-bac8-b392bc825df7 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:11.223195 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece5a1b-8b0e-443e-b174-132c2263cdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:11.223195 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece5a1b-8b0e-443e-b174-132c2263cdf6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:11.223554 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece5a1b-8b0e-443e-b174-132c2263cdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:11.223718 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece5a1b-8b0e-443e-b174-132c2263cdf6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:11.224540 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece5a1b-8b0e-443e-b174-132c2263cdf6 None admin] RBAC: Adding query filter params (interface=admin, service_id=e89f4b14b48c40a199cd99daa6b0d0f6, region_id=RegionOne) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:11.224848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece5a1b-8b0e-443e-b174-132c2263cdf6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:11.226252 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece5a1b-8b0e-443e-b174-132c2263cdf6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:11.233291 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 480/983] 10.222.0.121 () {56 vars in 1254 bytes} [Thu Aug 4 10:20:11 2022] GET /identity/v3/endpoints?service_id=e89f4b14b48c40a199cd99daa6b0d0f6&interface=admin®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:11.241243 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece5a1b-8b0e-443e-b174-132c2263cdf6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:11.245303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b8c02-d77a-4f7a-8c54-de588fd6fb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:11.245303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b8c02-d77a-4f7a-8c54-de588fd6fb14 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:11.246692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b8c02-d77a-4f7a-8c54-de588fd6fb14 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:11.246692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b8c02-d77a-4f7a-8c54-de588fd6fb14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:11.258129 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636b8c02-d77a-4f7a-8c54-de588fd6fb14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:11.258888 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636b8c02-d77a-4f7a-8c54-de588fd6fb14 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:11.275918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-636b8c02-d77a-4f7a-8c54-de588fd6fb14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:20:11.275918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:11.275918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:11.275918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:11.275918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:11.275918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:11.275918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:11.275918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:11.275918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:11.275918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:11.275918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:11.275918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:11.275918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:20:11.275918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:20:11.275918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:20:11.282249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:20:11.282249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:11.282249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:11.282249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:11.282249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:11.282249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:11.282249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:11.282249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:11.282249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:11.282249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:11.282249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:11.282249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:11.282249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:11.282249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:11.282249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:11.282249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:11.282249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:11.283212 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:20:11.283212 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:20:11.283212 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:20:11.283212 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:11.283212 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 504/984] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:20:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:11.283497 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b8c02-d77a-4f7a-8c54-de588fd6fb14 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:11.309552 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:11.385612 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece5a1b-8b0e-443e-b174-132c2263cdf6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:11.415985 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b8c02-d77a-4f7a-8c54-de588fd6fb14 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:11.426828 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ecdfa-eba5-4e46-902a-a24d7e74cf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:11.427518 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ecdfa-eba5-4e46-902a-a24d7e74cf22 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:11.427821 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ecdfa-eba5-4e46-902a-a24d7e74cf22 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:11.428681 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ecdfa-eba5-4e46-902a-a24d7e74cf22 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:11.429369 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ecdfa-eba5-4e46-902a-a24d7e74cf22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:11.430776 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ecdfa-eba5-4e46-902a-a24d7e74cf22 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:11.431734 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:11.447986 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 481/985] 10.222.0.121 () {56 vars in 1092 bytes} [Thu Aug 4 10:20:11 2022] GET /identity/v3/services => generated 1735 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:20:11.488145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c90f7-f897-4142-bb92-98f9abba0957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:11.488817 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c90f7-f897-4142-bb92-98f9abba0957 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:11.489130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c90f7-f897-4142-bb92-98f9abba0957 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:11.489370 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c90f7-f897-4142-bb92-98f9abba0957 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:11.490317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c90f7-f897-4142-bb92-98f9abba0957 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:11.490630 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c90f7-f897-4142-bb92-98f9abba0957 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:11.492226 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c90f7-f897-4142-bb92-98f9abba0957 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:11.510283 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 505/986] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:20:11 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:11.520080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ecdfa-eba5-4e46-902a-a24d7e74cf22 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:11.683016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ecdfa-eba5-4e46-902a-a24d7e74cf22 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:11.691353 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:11.726041 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6b24b-6278-4186-9762-7ff30b195214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:11.726701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6b24b-6278-4186-9762-7ff30b195214 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:11.726983 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6b24b-6278-4186-9762-7ff30b195214 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:11.727255 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6b24b-6278-4186-9762-7ff30b195214 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:11.734923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6b24b-6278-4186-9762-7ff30b195214 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:11.736726 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6b24b-6278-4186-9762-7ff30b195214 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:11.750376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-13f6b24b-6278-4186-9762-7ff30b195214 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:20:11.750376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:11.750376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:11.750376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:11.750376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:11.750376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:11.750376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:11.750376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:11.750376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:11.750376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:11.750376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:11.750376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:11.750376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:20:11.750376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:20:11.750376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:20:11.752202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:20:11.752202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:11.752202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:11.752202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:11.752202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:11.752202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:11.752202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:11.752202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:11.752202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:11.752202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:11.752202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:11.752202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:11.752202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:11.752202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:11.752202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:11.752202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:11.752202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:11.759577 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:20:11.759577 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:20:11.759577 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:20:11.759577 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:20:11.759577 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:20:11.759577 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:20:11.759577 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:20:11.759577 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:11.759577 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 482/987] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:20:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:11.760826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c90f7-f897-4142-bb92-98f9abba0957 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:11.899071 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c90f7-f897-4142-bb92-98f9abba0957 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:11.939909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49b7df-362f-4191-8556-d0c04ae701b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:11.940615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49b7df-362f-4191-8556-d0c04ae701b7 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:11.940900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49b7df-362f-4191-8556-d0c04ae701b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:11.941183 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49b7df-362f-4191-8556-d0c04ae701b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:11.942013 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49b7df-362f-4191-8556-d0c04ae701b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:11.942344 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49b7df-362f-4191-8556-d0c04ae701b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:11.943981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49b7df-362f-4191-8556-d0c04ae701b7 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:11.976052 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 506/988] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:20:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:11.979426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6b24b-6278-4186-9762-7ff30b195214 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:12.118948 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6b24b-6278-4186-9762-7ff30b195214 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:12.169027 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b5e27-e550-4dee-96d4-2f5b9c1e761a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:12.169661 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b5e27-e550-4dee-96d4-2f5b9c1e761a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:12.169953 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b5e27-e550-4dee-96d4-2f5b9c1e761a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:12.170232 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b5e27-e550-4dee-96d4-2f5b9c1e761a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:12.171030 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b5e27-e550-4dee-96d4-2f5b9c1e761a None admin] RBAC: Adding query filter params (role.id=9395de2ca6f34dfc85ea530fc0a14f45, scope.project.id=22768aba760b4307aab131d1312b2bf6, user.id=efe6e59228bd49bfbff25a11eb682426) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:12.171344 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b5e27-e550-4dee-96d4-2f5b9c1e761a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:12.172843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b5e27-e550-4dee-96d4-2f5b9c1e761a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:12.180000 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 483/989] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:20:11 2022] GET /identity/v3/role_assignments?user.id=efe6e59228bd49bfbff25a11eb682426&scope.project.id=22768aba760b4307aab131d1312b2bf6&role.id=9395de2ca6f34dfc85ea530fc0a14f45 => generated 610 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:12.808741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e531d-5da6-40ad-b334-3b8d629bd074 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:12.809397 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e531d-5da6-40ad-b334-3b8d629bd074 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:12.809397 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e531d-5da6-40ad-b334-3b8d629bd074 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:12.810869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 507/990] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:12.816545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e62beb-9204-4d51-b00e-ff4539ef954b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:12.817035 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e62beb-9204-4d51-b00e-ff4539ef954b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:12.817145 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e62beb-9204-4d51-b00e-ff4539ef954b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:12.838524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-57e62beb-9204-4d51-b00e-ff4539ef954b None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:12.839689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-57e62beb-9204-4d51-b00e-ff4539ef954b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:12.869773 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:12.926166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 484/991] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:12 2022] POST /identity/v3/auth/tokens => generated 1572 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:12.949092 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e531d-5da6-40ad-b334-3b8d629bd074 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:12.981731 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:13.089193 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e531d-5da6-40ad-b334-3b8d629bd074 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:13.154344 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8db27e-f4c0-4da6-b6bf-eb73b583ff6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:13.155429 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8db27e-f4c0-4da6-b6bf-eb73b583ff6b None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:13.155905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8db27e-f4c0-4da6-b6bf-eb73b583ff6b None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:13.156423 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8db27e-f4c0-4da6-b6bf-eb73b583ff6b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:13.157545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8db27e-f4c0-4da6-b6bf-eb73b583ff6b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:13.160272 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8db27e-f4c0-4da6-b6bf-eb73b583ff6b None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:13.178439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-0b8db27e-f4c0-4da6-b6bf-eb73b583ff6b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 10:20:13.178439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:13.178439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:20:13.178439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:20:13.178439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:20:13.178439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:20:13.178439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:20:13.178439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:20:13.178439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 10:20:13.178439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:13.178439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:20:13.178439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:13.178439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:13.178439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:13.178439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:13.178439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:13.178439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:13.181216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:13.181216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:13.181216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:13.181216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:13.181216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:13.181216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:13.181216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:20:13.181216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:20:13.181216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:20:13.181216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:20:13.181216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:13.181216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:13.181216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:13.181216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:13.181216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:13.181216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:13.183525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:13.183525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:13.183525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:13.183525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:13.183525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:13.183525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:13.183525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:13.183525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:13.183525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:13.183525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:13.183525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:20:13.183525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:20:13.183525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 10:20:13.183525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:13.187150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 508/992] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:20:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:13.191289 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e62beb-9204-4d51-b00e-ff4539ef954b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:13.388545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e62beb-9204-4d51-b00e-ff4539ef954b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:13.458866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75105655-7c21-4456-9496-71eae6748518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:13.459883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75105655-7c21-4456-9496-71eae6748518 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:13.460400 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75105655-7c21-4456-9496-71eae6748518 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:13.461139 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75105655-7c21-4456-9496-71eae6748518 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:13.462272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75105655-7c21-4456-9496-71eae6748518 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:13.462804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75105655-7c21-4456-9496-71eae6748518 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:13.465616 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75105655-7c21-4456-9496-71eae6748518 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:13.480179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 485/993] 10.222.0.121 () {56 vars in 1117 bytes} [Thu Aug 4 10:20:13 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:13.485635 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8db27e-f4c0-4da6-b6bf-eb73b583ff6b None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:13.542733 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdfd37e-5af7-43c8-ac1f-69465ce930cd None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:13.543764 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdfd37e-5af7-43c8-ac1f-69465ce930cd None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:13.544422 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdfd37e-5af7-43c8-ac1f-69465ce930cd None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:13.547277 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 486/994] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:13 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:13.552867 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87985c94-ed0f-4800-b373-85429d82f993 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:13.553149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87985c94-ed0f-4800-b373-85429d82f993 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:13.553410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87985c94-ed0f-4800-b373-85429d82f993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:13.575003 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-87985c94-ed0f-4800-b373-85429d82f993 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:13.576274 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-87985c94-ed0f-4800-b373-85429d82f993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:13.600760 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:13.638042 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:13.650739 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8db27e-f4c0-4da6-b6bf-eb73b583ff6b None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:13.684049 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 487/995] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:13 2022] POST /identity/v3/auth/tokens => generated 1572 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:20:13.698499 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61715b-4ca2-498b-98e5-3fe04d6920ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:13.699506 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61715b-4ca2-498b-98e5-3fe04d6920ea None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:13.699988 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61715b-4ca2-498b-98e5-3fe04d6920ea None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:13.700446 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61715b-4ca2-498b-98e5-3fe04d6920ea None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:13.701620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a61715b-4ca2-498b-98e5-3fe04d6920ea None admin] RBAC: Adding query filter params (type=compute) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:13.702143 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a61715b-4ca2-498b-98e5-3fe04d6920ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:13.702857 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87985c94-ed0f-4800-b373-85429d82f993 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:13.704637 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a61715b-4ca2-498b-98e5-3fe04d6920ea None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:13.713889 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 509/996] 10.222.0.121 () {56 vars in 1117 bytes} [Thu Aug 4 10:20:13 2022] GET /identity/v3/services?type=compute => generated 354 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:13.720930 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61715b-4ca2-498b-98e5-3fe04d6920ea None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:13.870630 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87985c94-ed0f-4800-b373-85429d82f993 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:13.889290 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61715b-4ca2-498b-98e5-3fe04d6920ea None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:13.912122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6447bc-254e-4c3d-83b4-9ae73f2a5fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:13.912684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6447bc-254e-4c3d-83b4-9ae73f2a5fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:13.912969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6447bc-254e-4c3d-83b4-9ae73f2a5fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:13.913245 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6447bc-254e-4c3d-83b4-9ae73f2a5fd8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:13.913894 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6447bc-254e-4c3d-83b4-9ae73f2a5fd8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:13.915303 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6447bc-254e-4c3d-83b4-9ae73f2a5fd8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:13.930166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-9e6447bc-254e-4c3d-83b4-9ae73f2a5fd8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:20:13.930166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:13.930166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:20:13.930166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:20:13.930166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:20:13.930166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:20:13.930166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:20:13.930166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:20:13.930166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:20:13.930166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:13.930166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:20:13.930166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:13.930166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:13.930166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:13.930166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:13.930166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:13.930166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:13.940883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:13.940883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:13.940883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:13.940883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:13.940883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:13.940883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:13.940883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:20:13.940883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:20:13.940883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:20:13.940883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:20:13.940883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:13.940883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:13.940883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:13.940883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:13.940883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:13.940883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:13.942596 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:13.942596 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:13.942596 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:13.942596 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:13.942596 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:13.942596 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:13.942596 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:13.942596 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:13.942596 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:13.942596 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:13.942596 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:20:13.942596 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:20:13.942596 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 10:20:13.942596 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:13.942596 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 488/997] 10.222.0.121 () {56 vars in 1131 bytes} [Thu Aug 4 10:20:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:13.946507 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6447bc-254e-4c3d-83b4-9ae73f2a5fd8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:13.958972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f15bd-fd62-46f6-9483-bc3d5473d714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:13.959633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f15bd-fd62-46f6-9483-bc3d5473d714 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:13.959925 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f15bd-fd62-46f6-9483-bc3d5473d714 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:13.960235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f15bd-fd62-46f6-9483-bc3d5473d714 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:13.961008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f15bd-fd62-46f6-9483-bc3d5473d714 None admin] RBAC: Adding query filter params (interface=public, service_id=110099b37179457585d57d7472d338fb, region_id=RegionOne) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:13.961334 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f15bd-fd62-46f6-9483-bc3d5473d714 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:13.963376 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f15bd-fd62-46f6-9483-bc3d5473d714 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:13.978064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 510/998] 10.222.0.121 () {56 vars in 1256 bytes} [Thu Aug 4 10:20:13 2022] GET /identity/v3/endpoints?service_id=110099b37179457585d57d7472d338fb&interface=public®ion_id=RegionOne => generated 195 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:13.979151 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f15bd-fd62-46f6-9483-bc3d5473d714 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:13.981935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:14.006916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:14.128274 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6447bc-254e-4c3d-83b4-9ae73f2a5fd8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:14.149953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f15bd-fd62-46f6-9483-bc3d5473d714 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:14.176495 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f84e0a7-01ab-4c63-a2f5-512025fab9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:14.177012 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84e0a7-01ab-4c63-a2f5-512025fab9ea None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:14.177342 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84e0a7-01ab-4c63-a2f5-512025fab9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:14.177616 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84e0a7-01ab-4c63-a2f5-512025fab9ea None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:14.178311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f84e0a7-01ab-4c63-a2f5-512025fab9ea None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:14.178624 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f84e0a7-01ab-4c63-a2f5-512025fab9ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:14.180075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f84e0a7-01ab-4c63-a2f5-512025fab9ea None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:14.185644 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 489/999] 10.222.0.121 () {56 vars in 1127 bytes} [Thu Aug 4 10:20:13 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:14.191859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f84e0a7-01ab-4c63-a2f5-512025fab9ea None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:14.220200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00167f6-33a9-4cb8-ba7b-a8b9e88d69fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:14.221967 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00167f6-33a9-4cb8-ba7b-a8b9e88d69fe None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:14.222791 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00167f6-33a9-4cb8-ba7b-a8b9e88d69fe None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:14.223543 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00167f6-33a9-4cb8-ba7b-a8b9e88d69fe None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:14.224959 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00167f6-33a9-4cb8-ba7b-a8b9e88d69fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:14.228006 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00167f6-33a9-4cb8-ba7b-a8b9e88d69fe None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:14.242077 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 511/1000] 10.222.0.121 () {56 vars in 1092 bytes} [Thu Aug 4 10:20:13 2022] GET /identity/v3/services => generated 1735 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:20:14.314746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:14.343192 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f84e0a7-01ab-4c63-a2f5-512025fab9ea None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:14.392026 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa773e31-ec3d-456c-80a0-460e1a55a070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:14.393000 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa773e31-ec3d-456c-80a0-460e1a55a070 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:14.393440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa773e31-ec3d-456c-80a0-460e1a55a070 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:14.393867 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa773e31-ec3d-456c-80a0-460e1a55a070 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:14.394770 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa773e31-ec3d-456c-80a0-460e1a55a070 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:14.395246 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa773e31-ec3d-456c-80a0-460e1a55a070 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:14.397003 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa773e31-ec3d-456c-80a0-460e1a55a070 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:14.404018 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 490/1001] 10.222.0.121 () {56 vars in 1127 bytes} [Thu Aug 4 10:20:14 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:14.410625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00167f6-33a9-4cb8-ba7b-a8b9e88d69fe None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:14.540201 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:14.568623 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00167f6-33a9-4cb8-ba7b-a8b9e88d69fe None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:14.620846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627e7d0-41a4-4a60-878c-a107d140c23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:14.621760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627e7d0-41a4-4a60-878c-a107d140c23b None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:14.622190 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627e7d0-41a4-4a60-878c-a107d140c23b None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:14.622609 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627e7d0-41a4-4a60-878c-a107d140c23b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:14.623515 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2627e7d0-41a4-4a60-878c-a107d140c23b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:14.625190 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2627e7d0-41a4-4a60-878c-a107d140c23b None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:14.652007 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 512/1002] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:20:14 2022] POST /identity/v3/endpoints => generated 327 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:20:14.671120 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c717691-68f3-47db-8983-36888e04e724 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:14.671646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c717691-68f3-47db-8983-36888e04e724 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:14.672204 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c717691-68f3-47db-8983-36888e04e724 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:14.676012 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 491/1003] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:14 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:14.685819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46efc6-6f62-46ac-831c-cc40d2b2d992 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:14.686252 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46efc6-6f62-46ac-831c-cc40d2b2d992 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:14.686797 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46efc6-6f62-46ac-831c-cc40d2b2d992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:14.717416 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-0f46efc6-6f62-46ac-831c-cc40d2b2d992 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:14.719355 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-0f46efc6-6f62-46ac-831c-cc40d2b2d992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:14.870521 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 513/1004] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:14 2022] POST /identity/v3/auth/tokens => generated 1720 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:14.894911 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c717691-68f3-47db-8983-36888e04e724 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:15.119002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c717691-68f3-47db-8983-36888e04e724 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:15.187619 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e58ae-6966-4359-9d74-f3c4dbdd7cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:15.188725 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e58ae-6966-4359-9d74-f3c4dbdd7cea None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:15.189225 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e58ae-6966-4359-9d74-f3c4dbdd7cea None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:15.189727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e58ae-6966-4359-9d74-f3c4dbdd7cea None admin] PATH_INFO: `/v3/services/image` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:15.191015 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506e58ae-6966-4359-9d74-f3c4dbdd7cea None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:15.194012 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506e58ae-6966-4359-9d74-f3c4dbdd7cea None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:15.213572 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-506e58ae-6966-4359-9d74-f3c4dbdd7cea None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 10:20:15.213572 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:15.213572 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:20:15.213572 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:20:15.213572 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:20:15.213572 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:20:15.213572 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:20:15.213572 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:20:15.213572 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 10:20:15.213572 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:15.213572 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:20:15.213572 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:15.213572 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:15.213572 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:15.213572 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:15.213572 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:15.213572 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:15.214950 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:15.214950 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:15.214950 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:15.214950 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:15.214950 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:15.214950 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:15.214950 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:20:15.214950 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:20:15.214950 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:20:15.214950 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:20:15.214950 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:15.214950 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:15.214950 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:15.214950 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:15.214950 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:15.214950 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:15.215937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:15.215937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:15.215937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:15.215937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:15.215937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:15.215937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:15.215937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:15.215937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:15.215937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:15.215937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:15.215937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:20:15.215937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:20:15.215937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 10:20:15.215937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:15.217606 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 492/1005] 10.222.0.121 () {56 vars in 1110 bytes} [Thu Aug 4 10:20:14 2022] GET /identity/v3/services/image => generated 86 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:15.225193 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46efc6-6f62-46ac-831c-cc40d2b2d992 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:15.382185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:15.427432 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46efc6-6f62-46ac-831c-cc40d2b2d992 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:15.471969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98619cb4-13e5-46ab-a0ec-706d33e6ce3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:15.472485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98619cb4-13e5-46ab-a0ec-706d33e6ce3a None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:15.472757 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98619cb4-13e5-46ab-a0ec-706d33e6ce3a None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:15.473019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98619cb4-13e5-46ab-a0ec-706d33e6ce3a None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:15.473743 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98619cb4-13e5-46ab-a0ec-706d33e6ce3a None admin] RBAC: Adding query filter params (name=image) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:15.474048 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98619cb4-13e5-46ab-a0ec-706d33e6ce3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:15.475558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98619cb4-13e5-46ab-a0ec-706d33e6ce3a None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:15.482100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 514/1006] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:20:15 2022] GET /identity/v3/services?name=image => generated 123 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:15.489347 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e58ae-6966-4359-9d74-f3c4dbdd7cea None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:15.507769 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:15.628982 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e58ae-6966-4359-9d74-f3c4dbdd7cea None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:15.669051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5cd6c7-e299-4a9f-b5a1-4bb176274948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:15.669683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cd6c7-e299-4a9f-b5a1-4bb176274948 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:15.669965 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cd6c7-e299-4a9f-b5a1-4bb176274948 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:15.670238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cd6c7-e299-4a9f-b5a1-4bb176274948 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:15.670925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5cd6c7-e299-4a9f-b5a1-4bb176274948 None admin] RBAC: Adding query filter params (type=image) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:15.671233 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5cd6c7-e299-4a9f-b5a1-4bb176274948 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:15.672671 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5cd6c7-e299-4a9f-b5a1-4bb176274948 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:15.678819 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 493/1007] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:20:15 2022] GET /identity/v3/services?type=image => generated 123 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:17.060080 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71349091-8681-4471-8267-9398806e0a49 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:17.060987 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71349091-8681-4471-8267-9398806e0a49 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:17.060987 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71349091-8681-4471-8267-9398806e0a49 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:17.064970 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 515/1008] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:17 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:17.067200 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798caaf-79a3-4103-8ee2-707c58ed067e None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:17.068162 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798caaf-79a3-4103-8ee2-707c58ed067e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:17.068340 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798caaf-79a3-4103-8ee2-707c58ed067e None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:17.071782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 494/1009] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:17 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:17.072687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7293b-50aa-427e-ba7b-1b11e73ee522 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:17.073237 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7293b-50aa-427e-ba7b-1b11e73ee522 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:17.073676 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7293b-50aa-427e-ba7b-1b11e73ee522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:17.081352 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415555d-0abc-4a45-b098-e15b60eab649 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:17.081847 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415555d-0abc-4a45-b098-e15b60eab649 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:17.083412 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415555d-0abc-4a45-b098-e15b60eab649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:17.095557 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-6ca7293b-50aa-427e-ba7b-1b11e73ee522 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:17.097109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6ca7293b-50aa-427e-ba7b-1b11e73ee522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:17.116407 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-3415555d-0abc-4a45-b098-e15b60eab649 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:17.118453 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3415555d-0abc-4a45-b098-e15b60eab649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:17.204038 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 516/1010] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:17 2022] POST /identity/v3/auth/tokens => generated 1720 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:20:17.228570 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7293b-50aa-427e-ba7b-1b11e73ee522 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:17.268103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 495/1011] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:17 2022] POST /identity/v3/auth/tokens => generated 1720 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:20:17.302722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415555d-0abc-4a45-b098-e15b60eab649 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:17.307566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:17.337886 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:17.409034 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7293b-50aa-427e-ba7b-1b11e73ee522 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:17.455021 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415555d-0abc-4a45-b098-e15b60eab649 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:17.476898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6a71d-3db8-429d-8448-d0ba84d611f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:17.478012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6a71d-3db8-429d-8448-d0ba84d611f7 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:17.478587 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6a71d-3db8-429d-8448-d0ba84d611f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:17.479087 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6a71d-3db8-429d-8448-d0ba84d611f7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:17.480293 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e6a71d-3db8-429d-8448-d0ba84d611f7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:17.483178 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e6a71d-3db8-429d-8448-d0ba84d611f7 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:17.498761 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2739ed14-eb3b-4563-889d-6a5521d95f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:17.499645 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739ed14-eb3b-4563-889d-6a5521d95f99 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:17.500113 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739ed14-eb3b-4563-889d-6a5521d95f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:17.500545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739ed14-eb3b-4563-889d-6a5521d95f99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:17.501938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2739ed14-eb3b-4563-889d-6a5521d95f99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:17.502955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-81e6a71d-3db8-429d-8448-d0ba84d611f7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 10:20:17.502955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:17.502955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:20:17.502955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:20:17.502955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:20:17.502955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:20:17.502955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:20:17.502955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:20:17.502955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 10:20:17.502955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:17.502955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:20:17.502955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:17.502955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:17.502955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:17.502955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:17.502955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:17.502955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:17.505291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:17.505291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:17.505291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:17.505291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:17.505291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:17.505291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:17.505291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:20:17.505291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:20:17.505291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:20:17.505291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:20:17.505291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:17.505291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:17.505291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:17.505291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:17.505291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:17.505291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:17.507092 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:17.507092 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:17.507092 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:17.507092 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:17.507092 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:17.507092 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:17.507092 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:17.507092 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:17.507092 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:17.507092 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:17.507092 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:20:17.507092 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:20:17.507092 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 10:20:17.507092 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:17.508677 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2739ed14-eb3b-4563-889d-6a5521d95f99 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:17.508844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 517/1012] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:20:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:20:17.512012 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 496/1013] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:20:17 2022] GET /identity/v3/domains/default => generated 200 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:17.514865 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6a71d-3db8-429d-8448-d0ba84d611f7 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:17.519472 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2739ed14-eb3b-4563-889d-6a5521d95f99 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:17.648501 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:17.700737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6a71d-3db8-429d-8448-d0ba84d611f7 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:17.706791 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2739ed14-eb3b-4563-889d-6a5521d95f99 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:17.749036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:17.754612 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ff6c6-4e78-48df-a3fd-34de998b85ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:17.756095 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ff6c6-4e78-48df-a3fd-34de998b85ff None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:17.756547 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ff6c6-4e78-48df-a3fd-34de998b85ff None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:17.757298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ff6c6-4e78-48df-a3fd-34de998b85ff None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:17.758219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730ff6c6-4e78-48df-a3fd-34de998b85ff None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:17.760063 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730ff6c6-4e78-48df-a3fd-34de998b85ff None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:17.763795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982b03f-940d-4f5e-ab7a-853193c92ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:17.764861 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982b03f-940d-4f5e-ab7a-853193c92ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:17.765341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982b03f-940d-4f5e-ab7a-853193c92ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:17.765839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982b03f-940d-4f5e-ab7a-853193c92ba2 None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:17.767135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9982b03f-940d-4f5e-ab7a-853193c92ba2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:17.767705 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9982b03f-940d-4f5e-ab7a-853193c92ba2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:17.770426 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9982b03f-940d-4f5e-ab7a-853193c92ba2 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:17.785245 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 518/1014] 10.222.0.121 () {56 vars in 1117 bytes} [Thu Aug 4 10:20:17 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:17.789191 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982b03f-940d-4f5e-ab7a-853193c92ba2 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:17.802668 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 497/1015] 10.222.0.121 () {60 vars in 1146 bytes} [Thu Aug 4 10:20:17 2022] POST /identity/v3/projects => generated 312 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:20:17.938214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982b03f-940d-4f5e-ab7a-853193c92ba2 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:17.983403 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07ef16-cc45-409c-825b-0d376cfcd2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:17.984415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07ef16-cc45-409c-825b-0d376cfcd2af None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:17.984842 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07ef16-cc45-409c-825b-0d376cfcd2af None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:17.985338 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07ef16-cc45-409c-825b-0d376cfcd2af None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:17.986257 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07ef16-cc45-409c-825b-0d376cfcd2af None admin] RBAC: Adding query filter params (type=compute) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:17.986737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07ef16-cc45-409c-825b-0d376cfcd2af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:17.988736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07ef16-cc45-409c-825b-0d376cfcd2af None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:18.001210 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 519/1016] 10.222.0.121 () {56 vars in 1117 bytes} [Thu Aug 4 10:20:17 2022] GET /identity/v3/services?type=compute => generated 354 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:18.003268 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ff6c6-4e78-48df-a3fd-34de998b85ff None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:18.155671 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ff6c6-4e78-48df-a3fd-34de998b85ff None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:18.193205 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:18.209441 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed2f7e-d0d4-4a71-bd49-7c9eceee3faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:18.210092 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed2f7e-d0d4-4a71-bd49-7c9eceee3faf None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:18.210379 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed2f7e-d0d4-4a71-bd49-7c9eceee3faf None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:18.210656 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed2f7e-d0d4-4a71-bd49-7c9eceee3faf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:18.211425 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed2f7e-d0d4-4a71-bd49-7c9eceee3faf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:18.212892 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed2f7e-d0d4-4a71-bd49-7c9eceee3faf None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:18.242413 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 498/1017] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:20:18 2022] POST /identity/v3/endpoints => generated 336 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:20:18.732131 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c64b77-3e9d-45de-bb09-48dab13fbc18 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:18.732846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c64b77-3e9d-45de-bb09-48dab13fbc18 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:18.733192 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c64b77-3e9d-45de-bb09-48dab13fbc18 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:18.736081 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 520/1018] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:18 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:18.744232 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4ef22-f8cf-4254-8e6b-7fbe024a619d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:18.744705 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4ef22-f8cf-4254-8e6b-7fbe024a619d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:18.744941 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4ef22-f8cf-4254-8e6b-7fbe024a619d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:18.771094 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6cf4ef22-f8cf-4254-8e6b-7fbe024a619d None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:18.772384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6cf4ef22-f8cf-4254-8e6b-7fbe024a619d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:18.868552 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 499/1019] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:18 2022] POST /identity/v3/auth/tokens => generated 1877 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:18.892830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c64b77-3e9d-45de-bb09-48dab13fbc18 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:18.971721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:19.035081 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c64b77-3e9d-45de-bb09-48dab13fbc18 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:19.079100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1759b1-3714-4bc4-92af-66b3c784b248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:19.079876 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1759b1-3714-4bc4-92af-66b3c784b248 None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:19.080199 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1759b1-3714-4bc4-92af-66b3c784b248 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:19.080464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1759b1-3714-4bc4-92af-66b3c784b248 None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:19.081214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1759b1-3714-4bc4-92af-66b3c784b248 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:19.082781 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1759b1-3714-4bc4-92af-66b3c784b248 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:19.111996 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 521/1020] 10.222.0.121 () {60 vars in 1146 bytes} [Thu Aug 4 10:20:18 2022] POST /identity/v3/services => generated 243 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:20:20.505645 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07154122-144b-48d1-a542-1c152d327790 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:20.506420 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07154122-144b-48d1-a542-1c152d327790 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:20.506420 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07154122-144b-48d1-a542-1c152d327790 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:20.507815 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 500/1021] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:20.513386 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324d2ec-2868-483e-9a56-307a15bc3332 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:20.513973 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324d2ec-2868-483e-9a56-307a15bc3332 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:20.513973 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324d2ec-2868-483e-9a56-307a15bc3332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:20.533672 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-e324d2ec-2868-483e-9a56-307a15bc3332 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:20.534888 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-e324d2ec-2868-483e-9a56-307a15bc3332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:20.620095 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 522/1022] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:20 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:20:20.643548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07154122-144b-48d1-a542-1c152d327790 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:20.652918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3001fe-9b3e-4190-8f31-2d6747944264 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:20.653409 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3001fe-9b3e-4190-8f31-2d6747944264 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:20.653892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3001fe-9b3e-4190-8f31-2d6747944264 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:20.656537 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 523/1023] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:20 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:20.665385 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6e4a9-a373-4b84-a0e5-d9610461910c None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:20.665869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6e4a9-a373-4b84-a0e5-d9610461910c None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:20.666229 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6e4a9-a373-4b84-a0e5-d9610461910c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:20.697711 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-95e6e4a9-a373-4b84-a0e5-d9610461910c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:20.699546 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-95e6e4a9-a373-4b84-a0e5-d9610461910c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:20.746198 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:20.788159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07154122-144b-48d1-a542-1c152d327790 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:20.789545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 524/1024] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:20 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:20:20.800093 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:20.814934 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e6e4a9-a373-4b84-a0e5-d9610461910c None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:20.838676 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a13ee-5f1c-4930-b392-12ef541fb995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:20.840031 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a13ee-5f1c-4930-b392-12ef541fb995 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:20.840031 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a13ee-5f1c-4930-b392-12ef541fb995 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:20.840031 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a13ee-5f1c-4930-b392-12ef541fb995 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:20.857600 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a13ee-5f1c-4930-b392-12ef541fb995 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:20:20.863966 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-a62a13ee-5f1c-4930-b392-12ef541fb995 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 10:20:20.863966 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:20.863966 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:20.863966 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:20.863966 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:20.863966 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:20.863966 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:20.863966 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:20.863966 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:20.863966 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:20.863966 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:20.863966 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:20.863966 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:20:20.863966 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:20:20.863966 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:20:20.863966 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:20:20.865356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:20:20.865356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:20:20.865356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:20:20.865356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:20:20.865356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:20.865356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:20.865356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:20.865356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:20.865356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:20.865356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:20.865356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:20.865356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:20.865356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:20.865356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:20.865356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:20.865356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:20.866679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:20.866679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:20.866679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:20.866679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:20.866679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:20:20.866679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:20:20.866679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:20:20.866679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:20:20.866679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:20:20.866679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:20:20.866679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 10:20:20.866679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:20.866679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 501/1025] 10.222.0.121 () {56 vars in 1125 bytes} [Thu Aug 4 10:20:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:20.871901 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a13ee-5f1c-4930-b392-12ef541fb995 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:20.956350 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e6e4a9-a373-4b84-a0e5-d9610461910c None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:20.994446 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a13ee-5f1c-4930-b392-12ef541fb995 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:20.996645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640196a3-303c-4c5d-af9a-8e341bb876d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:20.998253 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640196a3-303c-4c5d-af9a-8e341bb876d0 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:20.998543 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640196a3-303c-4c5d-af9a-8e341bb876d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:20.998838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640196a3-303c-4c5d-af9a-8e341bb876d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:21.000183 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640196a3-303c-4c5d-af9a-8e341bb876d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:21.001730 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640196a3-303c-4c5d-af9a-8e341bb876d0 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:21.004685 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 525/1026] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:20:20 2022] GET /identity/v3/domains/default => generated 200 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:21.011304 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640196a3-303c-4c5d-af9a-8e341bb876d0 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:21.038578 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98369e9f-1a03-4ace-b1d0-003cc77f75e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:21.039111 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98369e9f-1a03-4ace-b1d0-003cc77f75e3 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:21.039393 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98369e9f-1a03-4ace-b1d0-003cc77f75e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:21.039665 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98369e9f-1a03-4ace-b1d0-003cc77f75e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:21.040427 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98369e9f-1a03-4ace-b1d0-003cc77f75e3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:21.040735 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98369e9f-1a03-4ace-b1d0-003cc77f75e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:21.042155 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98369e9f-1a03-4ace-b1d0-003cc77f75e3 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:21.050994 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 502/1027] 10.222.0.121 () {56 vars in 1120 bytes} [Thu Aug 4 10:20:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:21.057467 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98369e9f-1a03-4ace-b1d0-003cc77f75e3 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:21.060775 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:21.090157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:21.131210 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640196a3-303c-4c5d-af9a-8e341bb876d0 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:21.170824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e346f1-3405-40d2-9f86-a1414dccd4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:21.171529 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e346f1-3405-40d2-9f86-a1414dccd4d5 None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:21.171817 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e346f1-3405-40d2-9f86-a1414dccd4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:21.172133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e346f1-3405-40d2-9f86-a1414dccd4d5 None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:21.172913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e346f1-3405-40d2-9f86-a1414dccd4d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:21.174532 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e346f1-3405-40d2-9f86-a1414dccd4d5 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:21.176165 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98369e9f-1a03-4ace-b1d0-003cc77f75e3 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:21.202257 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 526/1028] 10.222.0.121 () {60 vars in 1137 bytes} [Thu Aug 4 10:20:21 2022] POST /identity/v3/users => generated 286 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:20:21.220346 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364418d9-b589-40cc-9789-1b1c15b1fb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:21.221521 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364418d9-b589-40cc-9789-1b1c15b1fb62 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:21.222041 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364418d9-b589-40cc-9789-1b1c15b1fb62 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:21.222713 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364418d9-b589-40cc-9789-1b1c15b1fb62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:21.240770 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364418d9-b589-40cc-9789-1b1c15b1fb62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:21.242935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364418d9-b589-40cc-9789-1b1c15b1fb62 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:21.266364 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-364418d9-b589-40cc-9789-1b1c15b1fb62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:20:21.266364 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:21.266364 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:21.266364 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:21.266364 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:21.266364 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:21.266364 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:21.266364 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:21.266364 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:21.266364 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:21.266364 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:21.266364 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:21.266364 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:20:21.266364 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:20:21.266364 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:20:21.267543 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:20:21.267543 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:21.267543 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:21.267543 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:21.267543 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:21.267543 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:21.267543 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:21.267543 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:21.267543 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:21.267543 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:21.267543 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:21.267543 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:21.267543 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:21.267543 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:21.267543 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:21.267543 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:21.267543 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:21.268623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:20:21.268623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:20:21.268623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:20:21.268623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:21.275866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 503/1029] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:20:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:20:21.277991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e346f1-3405-40d2-9f86-a1414dccd4d5 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:21.431517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:21.444642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e346f1-3405-40d2-9f86-a1414dccd4d5 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:21.497083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a3f009-f2d1-4889-a605-d54cf70f1b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:21.497755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3f009-f2d1-4889-a605-d54cf70f1b28 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:21.498032 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3f009-f2d1-4889-a605-d54cf70f1b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:21.498311 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3f009-f2d1-4889-a605-d54cf70f1b28 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:21.499037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a3f009-f2d1-4889-a605-d54cf70f1b28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:21.499366 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a3f009-f2d1-4889-a605-d54cf70f1b28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:21.500898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a3f009-f2d1-4889-a605-d54cf70f1b28 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:21.515802 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 527/1030] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:20:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:21.522317 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364418d9-b589-40cc-9789-1b1c15b1fb62 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:21.639602 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:21.662075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364418d9-b589-40cc-9789-1b1c15b1fb62 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:21.699826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fe076-fd54-4a4b-9c56-911b14a8fb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:21.700435 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fe076-fd54-4a4b-9c56-911b14a8fb6d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:21.700720 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fe076-fd54-4a4b-9c56-911b14a8fb6d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:21.701004 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fe076-fd54-4a4b-9c56-911b14a8fb6d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:21.710628 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7fe076-fd54-4a4b-9c56-911b14a8fb6d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:21.712235 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7fe076-fd54-4a4b-9c56-911b14a8fb6d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:21.725484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-0c7fe076-fd54-4a4b-9c56-911b14a8fb6d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:20:21.725484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:21.725484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:21.725484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:21.725484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:21.725484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:21.725484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:21.725484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:21.725484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:21.725484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:21.725484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:21.725484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:21.725484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:20:21.725484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:20:21.725484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:20:21.725484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:20:21.726696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:21.726696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:21.726696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:20:21.726696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:20:21.726696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:20:21.726696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:20:21.726696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:21.726696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:21.726696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:21.726696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:21.726696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:21.726696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:21.726696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:21.726696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:21.726696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:21.726696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:21.726696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:21.726696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:21.728048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:21.728048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:21.728048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:20:21.728048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:20:21.728048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:20:21.728048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:20:21.728048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:20:21.728048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:20:21.728048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:20:21.728048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:21.728048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 504/1031] 10.222.0.121 () {56 vars in 1098 bytes} [Thu Aug 4 10:20:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:21.733112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a3f009-f2d1-4889-a605-d54cf70f1b28 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:21.855426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c82cb4-ffe4-4092-a474-69a1db5bc3f6 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:21.855780 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c82cb4-ffe4-4092-a474-69a1db5bc3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:21.855996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c82cb4-ffe4-4092-a474-69a1db5bc3f6 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:21.857791 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 505/1032] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:21 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 10:20:21.863700 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7678d-b08b-4597-ac08-e06e98198a3c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:21.864033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7678d-b08b-4597-ac08-e06e98198a3c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:21.864424 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7678d-b08b-4597-ac08-e06e98198a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:21.885357 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d1d7678d-b08b-4597-ac08-e06e98198a3c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:21.886747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d1d7678d-b08b-4597-ac08-e06e98198a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:21.931329 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a3f009-f2d1-4889-a605-d54cf70f1b28 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:21.976627 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 506/1033] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:21 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:20:21.986953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5cdb4d-7ba2-4703-aed0-7efb8e986013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:21.988222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5cdb4d-7ba2-4703-aed0-7efb8e986013 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:21.988632 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5cdb4d-7ba2-4703-aed0-7efb8e986013 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:21.988978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5cdb4d-7ba2-4703-aed0-7efb8e986013 None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:21.990200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5cdb4d-7ba2-4703-aed0-7efb8e986013 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:21.990576 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5cdb4d-7ba2-4703-aed0-7efb8e986013 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:21.992916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5cdb4d-7ba2-4703-aed0-7efb8e986013 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:21.993734 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7678d-b08b-4597-ac08-e06e98198a3c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:22.028227 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 528/1034] 10.222.0.121 () {56 vars in 1138 bytes} [Thu Aug 4 10:20:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:22.034496 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5cdb4d-7ba2-4703-aed0-7efb8e986013 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:22.053260 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:22.087812 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:22.136139 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7678d-b08b-4597-ac08-e06e98198a3c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:22.168728 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5cdb4d-7ba2-4703-aed0-7efb8e986013 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:22.184549 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda739a-d5f8-449b-8b85-ec8a8dd2d184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:22.185199 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda739a-d5f8-449b-8b85-ec8a8dd2d184 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:22.185482 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda739a-d5f8-449b-8b85-ec8a8dd2d184 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:22.185758 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda739a-d5f8-449b-8b85-ec8a8dd2d184 None admin] PATH_INFO: `/v3/services/image` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:22.186395 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda739a-d5f8-449b-8b85-ec8a8dd2d184 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:22.187800 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda739a-d5f8-449b-8b85-ec8a8dd2d184 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:22.197490 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-bbda739a-d5f8-449b-8b85-ec8a8dd2d184 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 10:20:22.197490 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:22.197490 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:20:22.197490 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:20:22.197490 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:20:22.197490 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:20:22.197490 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:20:22.197490 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:20:22.197490 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 10:20:22.197490 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:22.197490 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:20:22.197490 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:22.197490 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:22.197490 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:22.197490 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:22.197490 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:22.197490 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:22.198605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:22.198605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:22.198605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:22.198605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:22.198605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:22.198605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:22.198605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:20:22.198605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:20:22.198605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:20:22.198605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:20:22.198605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:22.198605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:22.198605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:22.198605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:22.198605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:22.198605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:22.199690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:22.199690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:22.199690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:22.199690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:22.199690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:22.199690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:22.199690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:22.199690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:22.199690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:22.199690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:22.199690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:20:22.199690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:20:22.199690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 10:20:22.199690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:22.199690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 507/1035] 10.222.0.121 () {56 vars in 1110 bytes} [Thu Aug 4 10:20:21 2022] GET /identity/v3/services/image => generated 86 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:22.204914 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda739a-d5f8-449b-8b85-ec8a8dd2d184 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:22.209163 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aafac6-9fce-4098-b3b5-84bb7f8802a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:22.209808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aafac6-9fce-4098-b3b5-84bb7f8802a1 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:22.210087 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aafac6-9fce-4098-b3b5-84bb7f8802a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:22.210361 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aafac6-9fce-4098-b3b5-84bb7f8802a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:22.292305 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aafac6-9fce-4098-b3b5-84bb7f8802a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:22.294095 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aafac6-9fce-4098-b3b5-84bb7f8802a1 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:22.311782 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:22.329838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda739a-d5f8-449b-8b85-ec8a8dd2d184 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:22.371853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aec976-c878-466f-9e36-c5c431b3a850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:22.372541 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aec976-c878-466f-9e36-c5c431b3a850 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:22.372828 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aec976-c878-466f-9e36-c5c431b3a850 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:22.373106 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aec976-c878-466f-9e36-c5c431b3a850 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:22.373830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aec976-c878-466f-9e36-c5c431b3a850 None admin] RBAC: Adding query filter params (name=image) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:22.374142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aec976-c878-466f-9e36-c5c431b3a850 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:22.375654 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aec976-c878-466f-9e36-c5c431b3a850 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:22.377275 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:22.390139 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 508/1036] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:20:22 2022] GET /identity/v3/services?name=image => generated 123 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:22.390668 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aec976-c878-466f-9e36-c5c431b3a850 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:22.423215 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 529/1037] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:20:22 2022] GET /identity/v3/auth/tokens => generated 1973 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 10:20:22.429048 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aafac6-9fce-4098-b3b5-84bb7f8802a1 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:22.508004 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aec976-c878-466f-9e36-c5c431b3a850 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:22.544447 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aafac6-9fce-4098-b3b5-84bb7f8802a1 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:22.547469 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705b5f4-0cff-4bff-b5c0-00d28c908071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:22.548145 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705b5f4-0cff-4bff-b5c0-00d28c908071 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:22.548439 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705b5f4-0cff-4bff-b5c0-00d28c908071 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:22.548711 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705b5f4-0cff-4bff-b5c0-00d28c908071 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:22.549403 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705b5f4-0cff-4bff-b5c0-00d28c908071 None admin] RBAC: Adding query filter params (type=image) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:22.549713 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705b5f4-0cff-4bff-b5c0-00d28c908071 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:22.551109 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705b5f4-0cff-4bff-b5c0-00d28c908071 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:22.564794 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 509/1038] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:20:22 2022] GET /identity/v3/services?type=image => generated 352 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:22.565049 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:22.566302 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705b5f4-0cff-4bff-b5c0-00d28c908071 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:22.589008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda7f26-ae78-4a7c-b54e-62b04acc9831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:22.589727 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda7f26-ae78-4a7c-b54e-62b04acc9831 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:22.590559 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda7f26-ae78-4a7c-b54e-62b04acc9831 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:22.590845 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda7f26-ae78-4a7c-b54e-62b04acc9831 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:22.603271 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda7f26-ae78-4a7c-b54e-62b04acc9831 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:22.604831 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda7f26-ae78-4a7c-b54e-62b04acc9831 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:22.623574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-0fda7f26-ae78-4a7c-b54e-62b04acc9831 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:20:22.623574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:22.623574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:22.623574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:22.623574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:22.623574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:22.623574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:22.623574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:22.623574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:22.623574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:22.623574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:22.623574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:22.623574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:20:22.623574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:20:22.623574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:20:22.624656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:20:22.624656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:22.624656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:22.624656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:22.624656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:22.624656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:22.624656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:22.624656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:22.624656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:22.624656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:22.624656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:22.624656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:22.624656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:22.624656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:22.624656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:22.624656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:22.624656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:22.625622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:20:22.625622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:20:22.625622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:20:22.625622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:22.629958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 530/1039] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:20:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:20:22.631488 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda7f26-ae78-4a7c-b54e-62b04acc9831 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:22.643420 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:22.690222 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705b5f4-0cff-4bff-b5c0-00d28c908071 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:22.730751 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85acb9a-ec0a-414a-b6a8-d7f4519e3655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:22.731437 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85acb9a-ec0a-414a-b6a8-d7f4519e3655 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:22.731710 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85acb9a-ec0a-414a-b6a8-d7f4519e3655 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:22.732028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85acb9a-ec0a-414a-b6a8-d7f4519e3655 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:22.732889 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85acb9a-ec0a-414a-b6a8-d7f4519e3655 None admin] RBAC: Adding query filter params (interface=public, service_id=23924d1f6d204792b9879e753c592659, region_id=RegionOne) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:22.733211 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85acb9a-ec0a-414a-b6a8-d7f4519e3655 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:22.734769 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85acb9a-ec0a-414a-b6a8-d7f4519e3655 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:22.741801 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 510/1040] 10.222.0.121 () {56 vars in 1256 bytes} [Thu Aug 4 10:20:22 2022] GET /identity/v3/endpoints?service_id=23924d1f6d204792b9879e753c592659&interface=public®ion_id=RegionOne => generated 195 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:22.748524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85acb9a-ec0a-414a-b6a8-d7f4519e3655 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:22.759203 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda7f26-ae78-4a7c-b54e-62b04acc9831 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:22.805062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab3a12-8c49-4b81-8477-dc19a3ef85b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:22.805610 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab3a12-8c49-4b81-8477-dc19a3ef85b9 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:22.805883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab3a12-8c49-4b81-8477-dc19a3ef85b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:22.806251 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab3a12-8c49-4b81-8477-dc19a3ef85b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:22.806995 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab3a12-8c49-4b81-8477-dc19a3ef85b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:22.807322 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab3a12-8c49-4b81-8477-dc19a3ef85b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:22.808970 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab3a12-8c49-4b81-8477-dc19a3ef85b9 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:22.827989 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 531/1041] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:20:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:22.829124 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab3a12-8c49-4b81-8477-dc19a3ef85b9 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:22.863805 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:22.878214 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85acb9a-ec0a-414a-b6a8-d7f4519e3655 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:22.912321 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:22.923446 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d135e3-f9d6-44c6-8543-0ce6c2d5ab5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:22.924192 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d135e3-f9d6-44c6-8543-0ce6c2d5ab5c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:22.924482 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d135e3-f9d6-44c6-8543-0ce6c2d5ab5c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:22.924765 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d135e3-f9d6-44c6-8543-0ce6c2d5ab5c None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:22.925449 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d135e3-f9d6-44c6-8543-0ce6c2d5ab5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:22.926908 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d135e3-f9d6-44c6-8543-0ce6c2d5ab5c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:22.934735 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 511/1042] 10.222.0.121 () {56 vars in 1092 bytes} [Thu Aug 4 10:20:22 2022] GET /identity/v3/services => generated 1966 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:20:22.973925 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab3a12-8c49-4b81-8477-dc19a3ef85b9 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:23.019134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c41243-d2e6-45b7-bc5e-253e9439a9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:23.019890 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c41243-d2e6-45b7-bc5e-253e9439a9da None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:23.020202 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c41243-d2e6-45b7-bc5e-253e9439a9da None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:23.020537 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c41243-d2e6-45b7-bc5e-253e9439a9da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:23.029416 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c41243-d2e6-45b7-bc5e-253e9439a9da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:23.031034 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c41243-d2e6-45b7-bc5e-253e9439a9da None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:23.044487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-18c41243-d2e6-45b7-bc5e-253e9439a9da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:20:23.044487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:23.044487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:23.044487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:23.044487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:23.044487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:23.044487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:23.044487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:23.044487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:23.044487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:23.044487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:23.044487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:23.044487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:20:23.044487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:20:23.044487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:20:23.045755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:20:23.045755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:23.045755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:23.045755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:23.045755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:23.045755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:23.045755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:23.045755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:23.045755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:23.045755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:23.045755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:23.045755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:23.045755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:23.045755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:23.045755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:23.045755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:23.045755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:23.047387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:20:23.047387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:20:23.047387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:20:23.047387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:20:23.047387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:20:23.047387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:20:23.047387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:20:23.047387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:23.047387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 532/1043] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:20:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:20:23.052554 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d135e3-f9d6-44c6-8543-0ce6c2d5ab5c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:23.198176 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d135e3-f9d6-44c6-8543-0ce6c2d5ab5c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:23.236715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff510b09-bac9-449b-b9a9-13c8e740fe8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:23.237658 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff510b09-bac9-449b-b9a9-13c8e740fe8f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:23.237948 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff510b09-bac9-449b-b9a9-13c8e740fe8f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:23.238209 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff510b09-bac9-449b-b9a9-13c8e740fe8f None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:23.238984 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff510b09-bac9-449b-b9a9-13c8e740fe8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:23.239290 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff510b09-bac9-449b-b9a9-13c8e740fe8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:23.240825 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff510b09-bac9-449b-b9a9-13c8e740fe8f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:23.260139 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 512/1044] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:20:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:23.264009 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c41243-d2e6-45b7-bc5e-253e9439a9da None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:23.366729 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:23.384337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c41243-d2e6-45b7-bc5e-253e9439a9da None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:23.419134 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff69d76-114a-4472-95e2-8581b19bb443 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:23.419552 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff69d76-114a-4472-95e2-8581b19bb443 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:23.419856 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff69d76-114a-4472-95e2-8581b19bb443 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:23.421436 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 513/1045] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:23 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 10:20:23.424532 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e5fcd-87ac-4b41-b263-7cbc940824c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:23.425190 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e5fcd-87ac-4b41-b263-7cbc940824c1 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:23.425478 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e5fcd-87ac-4b41-b263-7cbc940824c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:23.426095 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e5fcd-87ac-4b41-b263-7cbc940824c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:23.426638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e5fcd-87ac-4b41-b263-7cbc940824c1 None admin] RBAC: Adding query filter params (role.id=9395de2ca6f34dfc85ea530fc0a14f45, scope.project.id=22768aba760b4307aab131d1312b2bf6, user.id=0d947aeb749546309a8d4b7938087ba0) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:23.426951 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e5fcd-87ac-4b41-b263-7cbc940824c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:23.427744 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a704d-fece-4bbe-9541-161b5c9cae4d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:23.428082 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a704d-fece-4bbe-9541-161b5c9cae4d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:23.428374 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a704d-fece-4bbe-9541-161b5c9cae4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:23.428665 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e5fcd-87ac-4b41-b263-7cbc940824c1 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:23.436225 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 533/1046] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:20:23 2022] GET /identity/v3/role_assignments?user.id=0d947aeb749546309a8d4b7938087ba0&scope.project.id=22768aba760b4307aab131d1312b2bf6&role.id=9395de2ca6f34dfc85ea530fc0a14f45 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:23.450736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-866a704d-fece-4bbe-9541-161b5c9cae4d None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:23.451364 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-866a704d-fece-4bbe-9541-161b5c9cae4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:23.524309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:23.560249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 514/1047] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:23 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:20:23.579549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e5fcd-87ac-4b41-b263-7cbc940824c1 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:23.726742 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e5fcd-87ac-4b41-b263-7cbc940824c1 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:23.779380 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c826c-1130-409e-8361-3c8d69bd637a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:23.780631 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c826c-1130-409e-8361-3c8d69bd637a None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:23.781141 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c826c-1130-409e-8361-3c8d69bd637a None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:23.781806 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c826c-1130-409e-8361-3c8d69bd637a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:23.798550 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c826c-1130-409e-8361-3c8d69bd637a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:20:23.804838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-b05c826c-1130-409e-8361-3c8d69bd637a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:20:23.804838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:23.804838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:23.804838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:23.804838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:23.804838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:23.804838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:23.804838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:23.804838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:23.804838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:23.804838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:23.804838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:23.804838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:20:23.804838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:20:23.804838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:20:23.804838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:20:23.805945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:20:23.805945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:20:23.805945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:20:23.805945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:20:23.805945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:23.805945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:23.805945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:23.805945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:23.805945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:23.805945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:23.805945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:23.805945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:23.805945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:23.805945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:23.805945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:23.805945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:23.806860 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:23.806860 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:23.806860 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:23.806860 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:23.806860 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:20:23.806860 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:20:23.806860 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:20:23.806860 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:20:23.806860 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:20:23.806860 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:20:23.806860 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:20:23.806860 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:23.812044 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 534/1048] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:20:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:23.815394 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a704d-fece-4bbe-9541-161b5c9cae4d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:23.957649 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a704d-fece-4bbe-9541-161b5c9cae4d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:24.007939 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1583c-542d-4a3b-aa9e-e0b64dc783d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:24.008809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1583c-542d-4a3b-aa9e-e0b64dc783d0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:24.009241 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1583c-542d-4a3b-aa9e-e0b64dc783d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:24.009543 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1583c-542d-4a3b-aa9e-e0b64dc783d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:24.010540 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1583c-542d-4a3b-aa9e-e0b64dc783d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:24.010920 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1583c-542d-4a3b-aa9e-e0b64dc783d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:24.013090 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1583c-542d-4a3b-aa9e-e0b64dc783d0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:24.030845 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 515/1049] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:20:23 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:24.033026 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c826c-1130-409e-8361-3c8d69bd637a None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:24.050057 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:24.164943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c826c-1130-409e-8361-3c8d69bd637a None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:24.221507 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da47ef-d40e-4914-9f4e-b48680d23657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:24.222183 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da47ef-d40e-4914-9f4e-b48680d23657 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:24.222471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da47ef-d40e-4914-9f4e-b48680d23657 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:24.222754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da47ef-d40e-4914-9f4e-b48680d23657 None admin] PATH_INFO: `/v3/users/b1bf26cbb3be4f749e7bf24ad624bb2d` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:24.239629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da47ef-d40e-4914-9f4e-b48680d23657 None admin] RBAC: Authorizing `identity:get_user(user_id=b1bf26cbb3be4f749e7bf24ad624bb2d)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:24.241271 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da47ef-d40e-4914-9f4e-b48680d23657 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:24.243488 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 535/1050] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:20:24 2022] GET /identity/v3/users/b1bf26cbb3be4f749e7bf24ad624bb2d => generated 286 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:24.249700 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1583c-542d-4a3b-aa9e-e0b64dc783d0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:24.322710 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:24.391470 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1583c-542d-4a3b-aa9e-e0b64dc783d0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:24.428404 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09a198-6c7c-43f9-bf57-02701ba7999c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:24.429046 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09a198-6c7c-43f9-bf57-02701ba7999c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:24.429487 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09a198-6c7c-43f9-bf57-02701ba7999c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:24.429766 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09a198-6c7c-43f9-bf57-02701ba7999c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:24.508497 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09a198-6c7c-43f9-bf57-02701ba7999c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:24.510349 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09a198-6c7c-43f9-bf57-02701ba7999c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:24.574051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:24.648348 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 516/1051] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:20:24 2022] GET /identity/v3/auth/tokens => generated 1973 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:20:24.655497 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da47ef-d40e-4914-9f4e-b48680d23657 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:24.739598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:24.782864 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da47ef-d40e-4914-9f4e-b48680d23657 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:24.821955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332a6ba-86e2-4c9c-a8cd-ca9c06bf5f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:24.822658 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332a6ba-86e2-4c9c-a8cd-ca9c06bf5f79 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:24.822952 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332a6ba-86e2-4c9c-a8cd-ca9c06bf5f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:24.823225 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332a6ba-86e2-4c9c-a8cd-ca9c06bf5f79 None admin] PATH_INFO: `/v3/projects/1e05b319c6e747a09159c629e77db830` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:24.824562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332a6ba-86e2-4c9c-a8cd-ca9c06bf5f79 None admin] RBAC: Authorizing `identity:get_project(project_id=1e05b319c6e747a09159c629e77db830)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:24.826192 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332a6ba-86e2-4c9c-a8cd-ca9c06bf5f79 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:24.833504 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 536/1052] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:20:24 2022] GET /identity/v3/projects/1e05b319c6e747a09159c629e77db830 => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:24.835206 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09a198-6c7c-43f9-bf57-02701ba7999c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:24.953035 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09a198-6c7c-43f9-bf57-02701ba7999c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:24.991211 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5deaba-a83e-49d6-be97-b5c06d4d37ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:24.991925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5deaba-a83e-49d6-be97-b5c06d4d37ac None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:24.992251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5deaba-a83e-49d6-be97-b5c06d4d37ac None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:24.992507 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5deaba-a83e-49d6-be97-b5c06d4d37ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:24.993317 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5deaba-a83e-49d6-be97-b5c06d4d37ac None admin] RBAC: Adding query filter params (role.id=7342106a255b4bce8f150726602a8b95, scope.project.id=1e05b319c6e747a09159c629e77db830, user.id=b1bf26cbb3be4f749e7bf24ad624bb2d) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:24.993621 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5deaba-a83e-49d6-be97-b5c06d4d37ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:24.995112 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5deaba-a83e-49d6-be97-b5c06d4d37ac None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:25.002620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 517/1053] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:20:24 2022] GET /identity/v3/role_assignments?user.id=b1bf26cbb3be4f749e7bf24ad624bb2d&scope.project.id=1e05b319c6e747a09159c629e77db830&role.id=7342106a255b4bce8f150726602a8b95 => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:25.259611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd469fa7-065b-49c4-9961-9576a38adbab None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:25.259835 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd469fa7-065b-49c4-9961-9576a38adbab None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:25.260152 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd469fa7-065b-49c4-9961-9576a38adbab None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:25.261536 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 537/1054] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:25.268043 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c41689-1574-485d-99de-ea627d6b0f1b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:25.268377 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c41689-1574-485d-99de-ea627d6b0f1b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:25.268702 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c41689-1574-485d-99de-ea627d6b0f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:25.283429 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:25.290540 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b6c41689-1574-485d-99de-ea627d6b0f1b None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:25.292293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b6c41689-1574-485d-99de-ea627d6b0f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:25.380411 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 518/1055] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:25 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:25.401687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd469fa7-065b-49c4-9961-9576a38adbab None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:25.548710 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd469fa7-065b-49c4-9961-9576a38adbab None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:25.608191 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d044305-53c4-4543-b4df-44cb7dcf1450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:25.609151 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d044305-53c4-4543-b4df-44cb7dcf1450 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:25.609437 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d044305-53c4-4543-b4df-44cb7dcf1450 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:25.609718 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d044305-53c4-4543-b4df-44cb7dcf1450 None admin] PATH_INFO: `/v3/services/image` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:25.610407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d044305-53c4-4543-b4df-44cb7dcf1450 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:25.612062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d044305-53c4-4543-b4df-44cb7dcf1450 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:25.616633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:25.617657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702dd44-f5b4-48db-b060-96fcab840d9d None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:25.618174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702dd44-f5b4-48db-b060-96fcab840d9d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:25.618493 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702dd44-f5b4-48db-b060-96fcab840d9d None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:25.620472 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 519/1056] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:25 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:25.627816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-3d044305-53c4-4543-b4df-44cb7dcf1450 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 10:20:25.627816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:25.627816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:20:25.627816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:20:25.627816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:20:25.627816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:20:25.627816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:20:25.627816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:20:25.627816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 10:20:25.627816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:25.627816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:20:25.627816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:25.627816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:25.627816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:25.627816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:25.627816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:25.627816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:25.631021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:25.631021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:25.631021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:25.631021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:25.631021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:25.631021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:25.631021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:20:25.631021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:20:25.631021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:20:25.631021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:20:25.631021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:25.631021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:25.631021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:25.631021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:25.631021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:25.631021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:25.631924 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:25.631924 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:25.631924 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:25.631924 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:25.631924 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:25.631924 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:25.631924 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:25.631924 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:25.631924 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:25.631924 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:25.631924 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:20:25.631924 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:20:25.631924 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 10:20:25.631924 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:25.631924 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 538/1057] 10.222.0.121 () {56 vars in 1110 bytes} [Thu Aug 4 10:20:25 2022] GET /identity/v3/services/image => generated 86 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:25.632905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d27b8e-bff7-4a12-bc31-16929498f90f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:25.632905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d27b8e-bff7-4a12-bc31-16929498f90f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:25.632905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d27b8e-bff7-4a12-bc31-16929498f90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:25.636231 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d044305-53c4-4543-b4df-44cb7dcf1450 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:25.654499 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-52d27b8e-bff7-4a12-bc31-16929498f90f None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:25.655746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-52d27b8e-bff7-4a12-bc31-16929498f90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:25.731398 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 520/1058] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:25 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:25.755008 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d27b8e-bff7-4a12-bc31-16929498f90f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:25.773401 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d044305-53c4-4543-b4df-44cb7dcf1450 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:25.813791 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925e837-cf18-4df5-9a1d-3f056a3c7590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:25.814334 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925e837-cf18-4df5-9a1d-3f056a3c7590 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:25.814622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925e837-cf18-4df5-9a1d-3f056a3c7590 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:25.814898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925e837-cf18-4df5-9a1d-3f056a3c7590 None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:25.815613 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925e837-cf18-4df5-9a1d-3f056a3c7590 None admin] RBAC: Adding query filter params (name=image) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:25.815933 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925e837-cf18-4df5-9a1d-3f056a3c7590 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:25.817420 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925e837-cf18-4df5-9a1d-3f056a3c7590 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:25.823868 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 539/1059] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:20:25 2022] GET /identity/v3/services?name=image => generated 123 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:25.830860 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925e837-cf18-4df5-9a1d-3f056a3c7590 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:25.841685 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:25.884309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d27b8e-bff7-4a12-bc31-16929498f90f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:25.927401 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:25.933924 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c302e6-54aa-4b3d-b313-2ebfea8facfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:25.934902 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c302e6-54aa-4b3d-b313-2ebfea8facfd None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:25.935321 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c302e6-54aa-4b3d-b313-2ebfea8facfd None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:25.935750 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c302e6-54aa-4b3d-b313-2ebfea8facfd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:25.949563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c302e6-54aa-4b3d-b313-2ebfea8facfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:20:25.954756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-f2c302e6-54aa-4b3d-b313-2ebfea8facfd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 10:20:25.954756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:25.954756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:25.954756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:25.954756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:25.954756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:25.954756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:25.954756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:25.954756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:25.954756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:25.954756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:25.954756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:25.954756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:20:25.954756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:20:25.954756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:20:25.954756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:20:25.955831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:20:25.955831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:20:25.955831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:20:25.955831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:20:25.955831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:25.955831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:25.955831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:25.955831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:25.955831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:25.955831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:25.955831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:25.955831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:25.955831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:25.955831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:25.955831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:25.955831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:25.956781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:25.956781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:25.956781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:25.956781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:25.956781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:20:25.956781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:20:25.956781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:20:25.956781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:20:25.956781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:20:25.956781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:20:25.956781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 10:20:25.956781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:25.956781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 521/1060] 10.222.0.121 () {56 vars in 1125 bytes} [Thu Aug 4 10:20:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:20:25.964244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c302e6-54aa-4b3d-b313-2ebfea8facfd None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:25.985911 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925e837-cf18-4df5-9a1d-3f056a3c7590 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:26.026991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14649245-54aa-4f29-842b-597f9af12df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:26.027668 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14649245-54aa-4f29-842b-597f9af12df9 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:26.028027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14649245-54aa-4f29-842b-597f9af12df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:26.028269 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14649245-54aa-4f29-842b-597f9af12df9 None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:26.029108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14649245-54aa-4f29-842b-597f9af12df9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:26.030083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14649245-54aa-4f29-842b-597f9af12df9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:26.032128 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14649245-54aa-4f29-842b-597f9af12df9 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:26.046127 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 540/1061] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:20:25 2022] GET /identity/v3/services?type=image => generated 352 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:26.047839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14649245-54aa-4f29-842b-597f9af12df9 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:26.085918 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c302e6-54aa-4b3d-b313-2ebfea8facfd None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:26.115704 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:26.128421 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad9874-b09d-44e3-a4af-858ee7434ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:26.129058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad9874-b09d-44e3-a4af-858ee7434ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:26.129345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad9874-b09d-44e3-a4af-858ee7434ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:26.129637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad9874-b09d-44e3-a4af-858ee7434ce1 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:26.131544 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad9874-b09d-44e3-a4af-858ee7434ce1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:26.131855 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad9874-b09d-44e3-a4af-858ee7434ce1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:26.134077 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad9874-b09d-44e3-a4af-858ee7434ce1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:26.145254 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 522/1062] 10.222.0.121 () {56 vars in 1120 bytes} [Thu Aug 4 10:20:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:26.152793 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad9874-b09d-44e3-a4af-858ee7434ce1 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:26.161322 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14649245-54aa-4f29-842b-597f9af12df9 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:26.203590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:26.217499 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd259e0c-3a68-447e-947d-44ac1e397931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:26.218593 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd259e0c-3a68-447e-947d-44ac1e397931 None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:26.218884 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd259e0c-3a68-447e-947d-44ac1e397931 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:26.219163 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd259e0c-3a68-447e-947d-44ac1e397931 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:26.220536 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd259e0c-3a68-447e-947d-44ac1e397931 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:26.223225 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd259e0c-3a68-447e-947d-44ac1e397931 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:26.267988 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 541/1063] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:20:26 2022] POST /identity/v3/endpoints => generated 329 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:20:26.284499 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad9874-b09d-44e3-a4af-858ee7434ce1 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:26.342828 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b583074-8338-43f0-98ea-ac46ea6b5d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:26.343750 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b583074-8338-43f0-98ea-ac46ea6b5d05 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:26.344223 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b583074-8338-43f0-98ea-ac46ea6b5d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:26.344645 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b583074-8338-43f0-98ea-ac46ea6b5d05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:26.360681 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b583074-8338-43f0-98ea-ac46ea6b5d05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:26.363181 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b583074-8338-43f0-98ea-ac46ea6b5d05 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:26.382442 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-3b583074-8338-43f0-98ea-ac46ea6b5d05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:20:26.382442 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:26.382442 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:26.382442 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:26.382442 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:26.382442 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:26.382442 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:26.382442 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:26.382442 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:26.382442 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:26.382442 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:26.382442 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:26.382442 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:20:26.382442 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:20:26.382442 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:20:26.383615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:20:26.383615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:26.383615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:26.383615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:26.383615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:26.383615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:26.383615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:26.383615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:26.383615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:26.383615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:26.383615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:26.383615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:26.383615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:26.383615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:26.383615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:26.383615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:26.383615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:26.384785 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:20:26.384785 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:20:26.384785 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:20:26.384785 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:26.388636 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 523/1064] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:20:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:26.391113 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd259e0c-3a68-447e-947d-44ac1e397931 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:26.516326 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd259e0c-3a68-447e-947d-44ac1e397931 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:26.558781 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f4b1b-577c-43b3-adbc-2e9e019641bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:26.559457 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f4b1b-577c-43b3-adbc-2e9e019641bd None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:26.559751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f4b1b-577c-43b3-adbc-2e9e019641bd None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:26.560119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f4b1b-577c-43b3-adbc-2e9e019641bd None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:26.560856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f4b1b-577c-43b3-adbc-2e9e019641bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:26.561177 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f4b1b-577c-43b3-adbc-2e9e019641bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:26.562624 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f4b1b-577c-43b3-adbc-2e9e019641bd None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:26.576683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 542/1065] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:20:26 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:26.584784 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b583074-8338-43f0-98ea-ac46ea6b5d05 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:26.627470 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:26.705149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b583074-8338-43f0-98ea-ac46ea6b5d05 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:26.755687 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36b451-fd07-4945-859f-dbf61251484f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:26.756435 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36b451-fd07-4945-859f-dbf61251484f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:26.756719 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36b451-fd07-4945-859f-dbf61251484f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:26.756983 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36b451-fd07-4945-859f-dbf61251484f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:26.767770 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e36b451-fd07-4945-859f-dbf61251484f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:26.769086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e36b451-fd07-4945-859f-dbf61251484f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:26.782742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-7e36b451-fd07-4945-859f-dbf61251484f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:20:26.782742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:26.782742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:26.782742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:26.782742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:26.782742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:26.782742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:26.782742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:26.782742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:26.782742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:26.782742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:26.782742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:26.782742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:20:26.782742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:20:26.782742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:20:26.782742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:20:26.784798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:26.784798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:26.784798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:20:26.784798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:20:26.784798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:20:26.784798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:20:26.784798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:26.784798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:26.784798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:26.784798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:26.784798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:26.784798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:26.784798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:26.784798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:26.784798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:26.784798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:26.784798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:26.784798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:26.785875 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:26.785875 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:26.785875 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:20:26.785875 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:20:26.785875 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:20:26.785875 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:20:26.785875 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:20:26.785875 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:20:26.785875 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:20:26.785875 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:26.785875 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 524/1066] 10.222.0.121 () {56 vars in 1098 bytes} [Thu Aug 4 10:20:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:26.793108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f4b1b-577c-43b3-adbc-2e9e019641bd None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:26.855914 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:26.929001 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f4b1b-577c-43b3-adbc-2e9e019641bd None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:26.972621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd35ad4-1133-4c7f-80b8-b4aa03416d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:26.973635 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd35ad4-1133-4c7f-80b8-b4aa03416d13 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:26.974235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd35ad4-1133-4c7f-80b8-b4aa03416d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:26.974740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd35ad4-1133-4c7f-80b8-b4aa03416d13 None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:26.975746 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd35ad4-1133-4c7f-80b8-b4aa03416d13 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:26.976368 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd35ad4-1133-4c7f-80b8-b4aa03416d13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:26.977993 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd35ad4-1133-4c7f-80b8-b4aa03416d13 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:27.003389 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 543/1067] 10.222.0.121 () {56 vars in 1138 bytes} [Thu Aug 4 10:20:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:27.005459 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36b451-fd07-4945-859f-dbf61251484f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:27.118091 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:27.127239 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36b451-fd07-4945-859f-dbf61251484f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:27.167329 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac1f10-1f53-4eb3-bafd-687118e4ba7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:27.168016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac1f10-1f53-4eb3-bafd-687118e4ba7b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:27.168290 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac1f10-1f53-4eb3-bafd-687118e4ba7b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:27.168583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac1f10-1f53-4eb3-bafd-687118e4ba7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:27.180942 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac1f10-1f53-4eb3-bafd-687118e4ba7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:27.182417 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac1f10-1f53-4eb3-bafd-687118e4ba7b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:27.199309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-b1ac1f10-1f53-4eb3-bafd-687118e4ba7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:20:27.199309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:27.199309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:27.199309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:27.199309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:27.199309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:27.199309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:27.199309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:27.199309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:27.199309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:27.199309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:27.199309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:27.199309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:20:27.199309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:20:27.199309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:20:27.200727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:20:27.200727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:27.200727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:27.200727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:27.200727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:27.200727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:27.200727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:27.200727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:27.200727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:27.200727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:27.200727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:27.200727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:27.200727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:27.200727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:27.200727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:27.200727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:27.200727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:27.201798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:20:27.201798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:20:27.201798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:20:27.201798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:27.201798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 525/1068] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:20:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:27.208369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd35ad4-1133-4c7f-80b8-b4aa03416d13 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:27.256338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16185b29-a387-490e-8b4b-0f46f69c6df8 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:27.256626 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16185b29-a387-490e-8b4b-0f46f69c6df8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:27.256901 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16185b29-a387-490e-8b4b-0f46f69c6df8 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:27.258737 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 526/1069] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:27 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:27.264403 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f2266-e566-4b4b-a78b-03f31febf85c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:27.264691 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f2266-e566-4b4b-a78b-03f31febf85c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:27.264977 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f2266-e566-4b4b-a78b-03f31febf85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:27.286322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0e3f2266-e566-4b4b-a78b-03f31febf85c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:27.287695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0e3f2266-e566-4b4b-a78b-03f31febf85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:27.328940 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd35ad4-1133-4c7f-80b8-b4aa03416d13 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:27.331144 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:27.374157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 527/1070] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:27 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:20:27.379256 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c22be-e4e1-4b4a-bb11-d40454de3030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:27.380537 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c22be-e4e1-4b4a-bb11-d40454de3030 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:27.381115 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c22be-e4e1-4b4a-bb11-d40454de3030 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:27.381710 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c22be-e4e1-4b4a-bb11-d40454de3030 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:27.383190 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c22be-e4e1-4b4a-bb11-d40454de3030 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:27.383885 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c22be-e4e1-4b4a-bb11-d40454de3030 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:27.386751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c22be-e4e1-4b4a-bb11-d40454de3030 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:27.397171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f2266-e566-4b4b-a78b-03f31febf85c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:27.407426 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 544/1071] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:20:27 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:27.413923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c22be-e4e1-4b4a-bb11-d40454de3030 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:27.492138 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:27.518472 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f2266-e566-4b4b-a78b-03f31febf85c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:27.536362 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c22be-e4e1-4b4a-bb11-d40454de3030 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:27.563568 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a665734-19f5-4669-90e6-01936c56a5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:27.564491 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a665734-19f5-4669-90e6-01936c56a5f0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:27.564839 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a665734-19f5-4669-90e6-01936c56a5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:27.565193 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a665734-19f5-4669-90e6-01936c56a5f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:27.577341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13773e90-7c7b-44c6-8b55-f0226c1ffeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:27.577940 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a665734-19f5-4669-90e6-01936c56a5f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:20:27.578180 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13773e90-7c7b-44c6-8b55-f0226c1ffeb9 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:27.578546 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13773e90-7c7b-44c6-8b55-f0226c1ffeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:27.578888 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13773e90-7c7b-44c6-8b55-f0226c1ffeb9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:27.586318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-6a665734-19f5-4669-90e6-01936c56a5f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:20:27.586318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:27.586318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:27.586318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:27.586318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:27.586318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:27.586318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:27.586318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:27.586318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:27.586318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:27.586318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:27.586318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:27.586318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:20:27.586318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:20:27.586318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:20:27.586318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:20:27.587563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:20:27.587563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:20:27.587563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:20:27.587563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:20:27.587563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:27.587563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:27.587563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:27.587563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:27.587563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:27.587563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:27.587563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:27.587563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:27.587563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:27.587563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:27.587563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:27.587563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:27.588959 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:27.588959 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:27.588959 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:27.588959 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:27.588959 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:20:27.588959 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:20:27.588959 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:20:27.588959 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:20:27.588959 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:20:27.588959 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:20:27.588959 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:20:27.588959 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:27.588959 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 528/1072] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:20:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:20:27.589861 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13773e90-7c7b-44c6-8b55-f0226c1ffeb9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:27.591099 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13773e90-7c7b-44c6-8b55-f0226c1ffeb9 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:27.595222 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a665734-19f5-4669-90e6-01936c56a5f0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:27.604133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-13773e90-7c7b-44c6-8b55-f0226c1ffeb9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:20:27.604133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:27.604133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:27.604133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:27.604133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:27.604133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:27.604133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:27.604133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:27.604133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:27.604133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:27.604133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:27.604133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:27.604133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:20:27.604133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:20:27.604133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:20:27.606294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:20:27.606294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:27.606294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:27.606294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:27.606294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:27.606294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:27.606294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:27.606294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:27.606294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:27.606294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:27.606294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:27.606294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:27.606294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:27.606294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:27.606294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:27.606294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:27.606294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:27.608563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:20:27.608563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:20:27.608563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:20:27.608563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:20:27.608563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:20:27.608563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:20:27.608563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:20:27.608563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:27.610420 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 545/1073] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:20:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:20:27.611303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13773e90-7c7b-44c6-8b55-f0226c1ffeb9 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:27.648190 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:27.740116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a665734-19f5-4669-90e6-01936c56a5f0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:27.748420 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13773e90-7c7b-44c6-8b55-f0226c1ffeb9 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:27.785718 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7bd11-604f-4459-9e78-a07f7b99ffc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:27.786969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7bd11-604f-4459-9e78-a07f7b99ffc9 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:27.787312 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7bd11-604f-4459-9e78-a07f7b99ffc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:27.787768 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7bd11-604f-4459-9e78-a07f7b99ffc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:27.788732 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a7bd11-604f-4459-9e78-a07f7b99ffc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:27.789185 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a7bd11-604f-4459-9e78-a07f7b99ffc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:27.790911 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a7bd11-604f-4459-9e78-a07f7b99ffc9 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:27.791933 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba97b7-8f0b-451c-8b2a-9c98a5984368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:27.793000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba97b7-8f0b-451c-8b2a-9c98a5984368 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:27.793465 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba97b7-8f0b-451c-8b2a-9c98a5984368 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:27.794467 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba97b7-8f0b-451c-8b2a-9c98a5984368 None admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:27.795377 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba97b7-8f0b-451c-8b2a-9c98a5984368 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:27.795894 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba97b7-8f0b-451c-8b2a-9c98a5984368 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:27.797825 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba97b7-8f0b-451c-8b2a-9c98a5984368 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:27.800823 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:27.811531 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 529/1074] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:20:27 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:27.813512 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7bd11-604f-4459-9e78-a07f7b99ffc9 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:27.821732 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 546/1075] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:20:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:27.823011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba97b7-8f0b-451c-8b2a-9c98a5984368 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:27.924382 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:27.951191 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba97b7-8f0b-451c-8b2a-9c98a5984368 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:27.989476 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7bd11-604f-4459-9e78-a07f7b99ffc9 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:27.995213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40451063-ec5d-4893-8eb9-3e364b5c27df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:27.996558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40451063-ec5d-4893-8eb9-3e364b5c27df None admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:27.997182 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40451063-ec5d-4893-8eb9-3e364b5c27df None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:27.997617 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40451063-ec5d-4893-8eb9-3e364b5c27df None admin] PATH_INFO: `/v3/projects/22768aba760b4307aab131d1312b2bf6/users/0d947aeb749546309a8d4b7938087ba0/roles/9395de2ca6f34dfc85ea530fc0a14f45` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:28.000728 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40451063-ec5d-4893-8eb9-3e364b5c27df None admin] RBAC: Authorizing `identity:create_grant(project_id=22768aba760b4307aab131d1312b2bf6, user_id=0d947aeb749546309a8d4b7938087ba0, role_id=9395de2ca6f34dfc85ea530fc0a14f45)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:28.003227 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40451063-ec5d-4893-8eb9-3e364b5c27df None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:28.032007 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 547/1076] 10.222.0.121 () {58 vars in 1444 bytes} [Thu Aug 4 10:20:27 2022] PUT /identity/v3/projects/22768aba760b4307aab131d1312b2bf6/users/0d947aeb749546309a8d4b7938087ba0/roles/9395de2ca6f34dfc85ea530fc0a14f45 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:20:28.060204 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a66aaa-cf0d-4068-a88c-163b6193239d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:28.061127 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a66aaa-cf0d-4068-a88c-163b6193239d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:28.061514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a66aaa-cf0d-4068-a88c-163b6193239d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:28.061851 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a66aaa-cf0d-4068-a88c-163b6193239d None admin] PATH_INFO: `/v3/users/b1bf26cbb3be4f749e7bf24ad624bb2d` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:28.063334 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a66aaa-cf0d-4068-a88c-163b6193239d None admin] RBAC: Authorizing `identity:get_user(user_id=b1bf26cbb3be4f749e7bf24ad624bb2d)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:28.065196 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a66aaa-cf0d-4068-a88c-163b6193239d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:28.067154 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 530/1077] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:20:27 2022] GET /identity/v3/users/b1bf26cbb3be4f749e7bf24ad624bb2d => generated 286 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:28.075483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40451063-ec5d-4893-8eb9-3e364b5c27df None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:28.215218 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40451063-ec5d-4893-8eb9-3e364b5c27df None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:28.260162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6797604-dbea-48d7-b7a2-6da5a1303a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:28.260918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6797604-dbea-48d7-b7a2-6da5a1303a6d None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:28.261402 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6797604-dbea-48d7-b7a2-6da5a1303a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:28.261826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6797604-dbea-48d7-b7a2-6da5a1303a6d None admin] PATH_INFO: `/v3/projects/1e05b319c6e747a09159c629e77db830` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:28.263163 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6797604-dbea-48d7-b7a2-6da5a1303a6d None admin] RBAC: Authorizing `identity:get_project(project_id=1e05b319c6e747a09159c629e77db830)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:28.264829 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6797604-dbea-48d7-b7a2-6da5a1303a6d None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:28.266977 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 548/1078] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:20:28 2022] GET /identity/v3/projects/1e05b319c6e747a09159c629e77db830 => generated 312 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:28.273404 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a66aaa-cf0d-4068-a88c-163b6193239d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:28.350781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:28.393535 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a66aaa-cf0d-4068-a88c-163b6193239d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:28.433382 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eeeddf-abe7-41ec-8639-a6db9505135a None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:28.433931 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eeeddf-abe7-41ec-8639-a6db9505135a None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:28.434411 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eeeddf-abe7-41ec-8639-a6db9505135a None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:28.434661 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d2b34-38a4-4318-a9f2-de9ab9cb50bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:28.435439 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d2b34-38a4-4318-a9f2-de9ab9cb50bc None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:28.435898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d2b34-38a4-4318-a9f2-de9ab9cb50bc None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:28.436310 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d2b34-38a4-4318-a9f2-de9ab9cb50bc None admin] PATH_INFO: `/v3/projects/1e05b319c6e747a09159c629e77db830/users/b1bf26cbb3be4f749e7bf24ad624bb2d/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:28.437236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 549/1079] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:28 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:28.438676 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37d2b34-38a4-4318-a9f2-de9ab9cb50bc None admin] RBAC: Authorizing `identity:create_grant(project_id=1e05b319c6e747a09159c629e77db830, user_id=b1bf26cbb3be4f749e7bf24ad624bb2d, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:28.440420 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37d2b34-38a4-4318-a9f2-de9ab9cb50bc None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:28.445274 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d209e9-9ddc-4206-ba14-16db679a48a7 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:28.446044 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d209e9-9ddc-4206-ba14-16db679a48a7 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:28.446723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d209e9-9ddc-4206-ba14-16db679a48a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:28.456168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 531/1080] 10.222.0.121 () {58 vars in 1444 bytes} [Thu Aug 4 10:20:28 2022] PUT /identity/v3/projects/1e05b319c6e747a09159c629e77db830/users/b1bf26cbb3be4f749e7bf24ad624bb2d/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:20:28.478731 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-63d209e9-9ddc-4206-ba14-16db679a48a7 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:28.480176 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-63d209e9-9ddc-4206-ba14-16db679a48a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:28.484316 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:28.603990 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 550/1081] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:28 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:28.639753 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d2b34-38a4-4318-a9f2-de9ab9cb50bc None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:28.824724 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d2b34-38a4-4318-a9f2-de9ab9cb50bc None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:28.874980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:28.891110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6072b-baaf-4f62-97bf-ed7e79cad6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:28.892231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6072b-baaf-4f62-97bf-ed7e79cad6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:28.892721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6072b-baaf-4f62-97bf-ed7e79cad6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:28.893229 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6072b-baaf-4f62-97bf-ed7e79cad6b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:29.021722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6072b-baaf-4f62-97bf-ed7e79cad6b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:29.024476 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6072b-baaf-4f62-97bf-ed7e79cad6b8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:29.167992 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 532/1082] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:20:28 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:20:29.174351 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d209e9-9ddc-4206-ba14-16db679a48a7 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:29.293825 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d209e9-9ddc-4206-ba14-16db679a48a7 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:29.337646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a883759-1118-4c3f-86cd-97685b2aa9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:29.338448 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a883759-1118-4c3f-86cd-97685b2aa9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:29.338864 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a883759-1118-4c3f-86cd-97685b2aa9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:29.339153 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a883759-1118-4c3f-86cd-97685b2aa9f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:29.351368 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:29.354787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a883759-1118-4c3f-86cd-97685b2aa9f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:29.356419 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a883759-1118-4c3f-86cd-97685b2aa9f4 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:29.375138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-5a883759-1118-4c3f-86cd-97685b2aa9f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:20:29.375138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:29.375138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:29.375138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:29.375138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:29.375138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:29.375138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:29.375138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:29.375138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:29.375138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:29.375138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:29.375138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:29.375138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:20:29.375138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:20:29.375138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:20:29.382133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:20:29.382133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:29.382133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:29.382133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:29.382133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:29.382133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:29.382133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:29.382133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:29.382133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:29.382133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:29.382133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:29.382133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:29.382133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:29.382133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:29.382133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:29.382133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:29.382133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:29.383116 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:20:29.383116 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:20:29.383116 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:20:29.383116 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:29.383116 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 551/1083] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:20:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:29.385058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6072b-baaf-4f62-97bf-ed7e79cad6b8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:29.418408 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:29.507685 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6072b-baaf-4f62-97bf-ed7e79cad6b8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:29.546824 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b14a2-0f5d-4b2e-9e16-9430524177bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:29.547492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b14a2-0f5d-4b2e-9e16-9430524177bb None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:29.547770 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b14a2-0f5d-4b2e-9e16-9430524177bb None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:29.548078 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b14a2-0f5d-4b2e-9e16-9430524177bb None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:29.548791 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b14a2-0f5d-4b2e-9e16-9430524177bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:29.549099 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b14a2-0f5d-4b2e-9e16-9430524177bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:29.550513 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b14a2-0f5d-4b2e-9e16-9430524177bb None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:29.569707 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 533/1084] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:20:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:30.202055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daef384-3092-46de-85b0-b3f6e9c658a4 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:30.202895 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daef384-3092-46de-85b0-b3f6e9c658a4 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:30.202895 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daef384-3092-46de-85b0-b3f6e9c658a4 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:30.204421 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 552/1085] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:30 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:30.210862 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b6b53-23cb-434b-b095-6d6e72ca3cae None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:30.211618 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b6b53-23cb-434b-b095-6d6e72ca3cae None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:30.211618 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b6b53-23cb-434b-b095-6d6e72ca3cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:30.234404 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e05b6b53-23cb-434b-b095-6d6e72ca3cae None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:30.235711 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e05b6b53-23cb-434b-b095-6d6e72ca3cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:30.317955 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 534/1086] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:30 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:30.342687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daef384-3092-46de-85b0-b3f6e9c658a4 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:30.495430 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daef384-3092-46de-85b0-b3f6e9c658a4 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:30.535100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96753050-6b30-4964-af96-7de1398a7b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:30.535778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96753050-6b30-4964-af96-7de1398a7b29 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:30.536103 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96753050-6b30-4964-af96-7de1398a7b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:30.536386 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96753050-6b30-4964-af96-7de1398a7b29 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:30.549902 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96753050-6b30-4964-af96-7de1398a7b29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:20:30.555160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-96753050-6b30-4964-af96-7de1398a7b29 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 10:20:30.555160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:30.555160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:30.555160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:30.555160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:30.555160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:30.555160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:30.555160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:30.555160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:30.555160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:30.555160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:30.555160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:30.555160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:20:30.555160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:20:30.555160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:20:30.555160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:20:30.563856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:20:30.563856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:20:30.563856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:20:30.563856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:20:30.563856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:30.563856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:30.563856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:30.563856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:30.563856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:30.563856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:30.563856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:30.563856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:30.563856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:30.563856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:30.563856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:30.563856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:30.565616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:30.565616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:30.565616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:30.565616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:30.565616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:20:30.565616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:20:30.565616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:20:30.565616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:20:30.565616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:20:30.565616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:20:30.565616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 10:20:30.565616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:30.565616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 553/1087] 10.222.0.121 () {56 vars in 1125 bytes} [Thu Aug 4 10:20:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:30.568296 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b6b53-23cb-434b-b095-6d6e72ca3cae None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:30.590274 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:30.727906 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b6b53-23cb-434b-b095-6d6e72ca3cae None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:30.770478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a41c20-1ca0-4045-98e8-d77f3a43554d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:30.771005 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a41c20-1ca0-4045-98e8-d77f3a43554d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:30.771284 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a41c20-1ca0-4045-98e8-d77f3a43554d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:30.771555 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a41c20-1ca0-4045-98e8-d77f3a43554d None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:30.772337 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a41c20-1ca0-4045-98e8-d77f3a43554d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:30.772641 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a41c20-1ca0-4045-98e8-d77f3a43554d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:30.774049 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a41c20-1ca0-4045-98e8-d77f3a43554d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:30.785047 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 535/1088] 10.222.0.121 () {56 vars in 1120 bytes} [Thu Aug 4 10:20:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:30.792080 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96753050-6b30-4964-af96-7de1398a7b29 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:30.857374 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:30.927921 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96753050-6b30-4964-af96-7de1398a7b29 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:30.979323 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7eda27-3fab-4b4d-b05a-6f0ac99d8b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:30.980056 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7eda27-3fab-4b4d-b05a-6f0ac99d8b18 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:30.980349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7eda27-3fab-4b4d-b05a-6f0ac99d8b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:30.980624 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7eda27-3fab-4b4d-b05a-6f0ac99d8b18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:30.994387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7eda27-3fab-4b4d-b05a-6f0ac99d8b18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:30.995976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7eda27-3fab-4b4d-b05a-6f0ac99d8b18 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:31.014814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-bd7eda27-3fab-4b4d-b05a-6f0ac99d8b18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:20:31.014814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:31.014814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:31.014814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:31.014814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:31.014814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:31.014814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:31.014814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:31.014814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:31.014814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:31.014814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:31.014814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:31.014814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:20:31.014814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:20:31.014814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:20:31.016431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:20:31.016431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:31.016431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:31.016431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:31.016431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:31.016431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:31.016431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:31.016431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:31.016431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:31.016431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:31.016431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:31.016431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:31.016431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:31.016431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:31.016431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:31.016431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:31.016431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:31.018923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:20:31.018923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:20:31.018923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:20:31.018923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:31.018923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 554/1089] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:20:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:31.022891 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a41c20-1ca0-4045-98e8-d77f3a43554d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:31.114289 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6be33-682d-4152-b58c-a29b5020c64a None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:31.114593 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6be33-682d-4152-b58c-a29b5020c64a None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:31.114984 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6be33-682d-4152-b58c-a29b5020c64a None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:31.116457 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 555/1090] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 10:20:31.118854 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:31.122430 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69dfb1b-f572-4ca7-b4d5-f6a785bc3c5f None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:31.122712 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69dfb1b-f572-4ca7-b4d5-f6a785bc3c5f None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:31.122990 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69dfb1b-f572-4ca7-b4d5-f6a785bc3c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:31.142016 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-a69dfb1b-f572-4ca7-b4d5-f6a785bc3c5f None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:31.143270 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-a69dfb1b-f572-4ca7-b4d5-f6a785bc3c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:31.148101 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a41c20-1ca0-4045-98e8-d77f3a43554d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:31.186635 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f794bf6d-e46e-45bc-aeaa-54bc5cfa924a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:31.187305 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794bf6d-e46e-45bc-aeaa-54bc5cfa924a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:31.187911 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794bf6d-e46e-45bc-aeaa-54bc5cfa924a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:31.187911 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794bf6d-e46e-45bc-aeaa-54bc5cfa924a None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:31.188596 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f794bf6d-e46e-45bc-aeaa-54bc5cfa924a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:31.188913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f794bf6d-e46e-45bc-aeaa-54bc5cfa924a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:31.190343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f794bf6d-e46e-45bc-aeaa-54bc5cfa924a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:31.205344 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 536/1091] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:20:31 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:31.213030 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f794bf6d-e46e-45bc-aeaa-54bc5cfa924a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:31.232911 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 556/1092] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:31 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:31.257877 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69dfb1b-f572-4ca7-b4d5-f6a785bc3c5f None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:31.297441 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:31.343478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f794bf6d-e46e-45bc-aeaa-54bc5cfa924a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:31.377343 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69dfb1b-f572-4ca7-b4d5-f6a785bc3c5f None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:31.388905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d0c5e-7a3d-4c2b-b15c-d8b752a7e81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:31.389698 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d0c5e-7a3d-4c2b-b15c-d8b752a7e81b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:31.389986 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d0c5e-7a3d-4c2b-b15c-d8b752a7e81b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:31.390278 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d0c5e-7a3d-4c2b-b15c-d8b752a7e81b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:31.398430 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d0c5e-7a3d-4c2b-b15c-d8b752a7e81b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:31.400060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d0c5e-7a3d-4c2b-b15c-d8b752a7e81b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:31.403030 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:31.422922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-0a9d0c5e-7a3d-4c2b-b15c-d8b752a7e81b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:20:31.422922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:31.422922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:31.422922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:31.422922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:31.422922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:31.422922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:31.422922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:31.422922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:31.422922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:31.422922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:31.422922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:31.422922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:20:31.422922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:20:31.422922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:20:31.422922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:20:31.424253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:31.424253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:31.424253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:20:31.424253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:20:31.424253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:20:31.424253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:20:31.424253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:31.424253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:31.424253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:31.424253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:31.424253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:31.424253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:31.424253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:31.424253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:31.424253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:31.424253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:31.424253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:31.424253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:31.425734 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:31.425734 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:31.425734 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:20:31.425734 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:20:31.425734 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:20:31.425734 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:20:31.425734 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:20:31.425734 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:20:31.425734 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 10:20:31.425734 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:31.426307 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a195fd-ae10-426d-a519-4a6626c149eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:31.426307 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a195fd-ae10-426d-a519-4a6626c149eb None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:31.426307 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a195fd-ae10-426d-a519-4a6626c149eb None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:31.426548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 537/1093] 10.222.0.121 () {56 vars in 1098 bytes} [Thu Aug 4 10:20:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:20:31.426635 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a195fd-ae10-426d-a519-4a6626c149eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:31.430379 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d0c5e-7a3d-4c2b-b15c-d8b752a7e81b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:31.438407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a195fd-ae10-426d-a519-4a6626c149eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:20:31.443562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-e0a195fd-ae10-426d-a519-4a6626c149eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:20:31.443562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:31.443562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:31.443562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:31.443562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:31.443562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:31.443562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:31.443562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:31.443562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:31.443562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:31.443562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:31.443562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:31.443562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:20:31.443562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:20:31.443562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:20:31.443562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:20:31.444721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:20:31.444721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:20:31.444721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:20:31.444721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:20:31.444721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:31.444721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:31.444721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:31.444721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:31.444721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:31.444721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:31.444721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:31.444721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:31.444721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:31.444721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:31.444721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:31.444721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:31.445744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:31.445744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:31.445744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:31.445744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:31.445744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:20:31.445744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:20:31.445744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:20:31.445744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:20:31.445744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:20:31.445744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:20:31.445744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:20:31.445744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:31.445744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 557/1094] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:20:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:31.451662 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a195fd-ae10-426d-a519-4a6626c149eb None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:31.549250 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d0c5e-7a3d-4c2b-b15c-d8b752a7e81b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:31.570258 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:31.579587 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a195fd-ae10-426d-a519-4a6626c149eb None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:31.593892 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614a494-b1cb-4d98-9949-fa217d048789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:31.594781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614a494-b1cb-4d98-9949-fa217d048789 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:31.594913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614a494-b1cb-4d98-9949-fa217d048789 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:31.595101 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614a494-b1cb-4d98-9949-fa217d048789 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:31.595915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614a494-b1cb-4d98-9949-fa217d048789 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:31.596256 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614a494-b1cb-4d98-9949-fa217d048789 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:31.597748 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614a494-b1cb-4d98-9949-fa217d048789 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:31.617339 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 538/1095] 10.222.0.121 () {56 vars in 1138 bytes} [Thu Aug 4 10:20:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:31.623211 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a1447-8def-46fc-b0b3-45c72a47a4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:31.623401 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614a494-b1cb-4d98-9949-fa217d048789 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:31.623860 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a1447-8def-46fc-b0b3-45c72a47a4a1 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:31.624169 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a1447-8def-46fc-b0b3-45c72a47a4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:31.624480 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a1447-8def-46fc-b0b3-45c72a47a4a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:31.625194 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a1447-8def-46fc-b0b3-45c72a47a4a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:31.625508 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a1447-8def-46fc-b0b3-45c72a47a4a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:31.626912 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a1447-8def-46fc-b0b3-45c72a47a4a1 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:31.637287 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 558/1096] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:20:31 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:31.643898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a1447-8def-46fc-b0b3-45c72a47a4a1 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:31.699343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:31.750119 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614a494-b1cb-4d98-9949-fa217d048789 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:31.769462 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a1447-8def-46fc-b0b3-45c72a47a4a1 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:31.804462 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9678c76-d9fb-435a-9033-94013bb247a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:31.805030 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9678c76-d9fb-435a-9033-94013bb247a6 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:31.805315 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9678c76-d9fb-435a-9033-94013bb247a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:31.805621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9678c76-d9fb-435a-9033-94013bb247a6 None admin] PATH_INFO: `/v3/users/b1bf26cbb3be4f749e7bf24ad624bb2d` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:31.806853 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9678c76-d9fb-435a-9033-94013bb247a6 None admin] RBAC: Authorizing `identity:get_user(user_id=b1bf26cbb3be4f749e7bf24ad624bb2d)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:31.808357 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9678c76-d9fb-435a-9033-94013bb247a6 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:31.809012 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87efe21-f35d-43d1-8954-302bbca63218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:31.810037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 559/1097] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:20:31 2022] GET /identity/v3/users/b1bf26cbb3be4f749e7bf24ad624bb2d => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:31.810200 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87efe21-f35d-43d1-8954-302bbca63218 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:31.810577 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87efe21-f35d-43d1-8954-302bbca63218 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:31.811076 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87efe21-f35d-43d1-8954-302bbca63218 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:31.816227 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9678c76-d9fb-435a-9033-94013bb247a6 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:31.844443 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:31.899882 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87efe21-f35d-43d1-8954-302bbca63218 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:31.901550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87efe21-f35d-43d1-8954-302bbca63218 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:31.926381 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9678c76-d9fb-435a-9033-94013bb247a6 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:31.960636 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:31.962805 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e5b8a-ce4c-4b74-aaae-38d7a644e8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:31.963503 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e5b8a-ce4c-4b74-aaae-38d7a644e8ab None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:31.963789 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e5b8a-ce4c-4b74-aaae-38d7a644e8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:31.964104 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e5b8a-ce4c-4b74-aaae-38d7a644e8ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:32.008158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 539/1098] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:20:31 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:20:32.014413 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87efe21-f35d-43d1-8954-302bbca63218 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:32.042801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e5b8a-ce4c-4b74-aaae-38d7a644e8ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:32.044326 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e5b8a-ce4c-4b74-aaae-38d7a644e8ab None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:32.071500 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:32.128071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87efe21-f35d-43d1-8954-302bbca63218 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:32.167935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7db46c-12af-4bd3-9c67-ae35df9bd309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:32.168624 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7db46c-12af-4bd3-9c67-ae35df9bd309 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:32.168904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7db46c-12af-4bd3-9c67-ae35df9bd309 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:32.169184 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7db46c-12af-4bd3-9c67-ae35df9bd309 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:32.170321 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 560/1099] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:20:31 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 10:20:32.176490 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e5b8a-ce4c-4b74-aaae-38d7a644e8ab None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:32.180328 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7db46c-12af-4bd3-9c67-ae35df9bd309 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:32.181847 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7db46c-12af-4bd3-9c67-ae35df9bd309 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:32.198905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-ff7db46c-12af-4bd3-9c67-ae35df9bd309 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:20:32.198905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:32.198905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:32.198905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:32.198905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:32.198905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:32.198905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:32.198905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:32.198905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:32.198905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:32.198905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:32.198905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:32.198905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:20:32.198905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:20:32.198905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:20:32.200045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:20:32.200045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:32.200045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:32.200045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:32.200045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:32.200045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:32.200045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:32.200045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:32.200045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:32.200045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:32.200045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:32.200045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:32.200045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:32.200045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:32.200045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:32.200045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:32.200045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:32.201027 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:20:32.201027 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:20:32.201027 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:20:32.201027 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:32.201027 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 540/1100] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:20:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:20:32.206320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7db46c-12af-4bd3-9c67-ae35df9bd309 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:32.222122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:32.288126 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e5b8a-ce4c-4b74-aaae-38d7a644e8ab None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:32.313677 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:32.327303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc82143-f819-4573-b307-6ad68095a8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:32.327534 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7db46c-12af-4bd3-9c67-ae35df9bd309 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:32.327887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc82143-f819-4573-b307-6ad68095a8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:32.328204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc82143-f819-4573-b307-6ad68095a8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:32.328481 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc82143-f819-4573-b307-6ad68095a8e7 None admin] PATH_INFO: `/v3/projects/1e05b319c6e747a09159c629e77db830` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:32.329793 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc82143-f819-4573-b307-6ad68095a8e7 None admin] RBAC: Authorizing `identity:get_project(project_id=1e05b319c6e747a09159c629e77db830)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:32.331236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc82143-f819-4573-b307-6ad68095a8e7 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:32.333009 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 561/1101] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:20:32 2022] GET /identity/v3/projects/1e05b319c6e747a09159c629e77db830 => generated 312 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:32.339502 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc82143-f819-4573-b307-6ad68095a8e7 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:32.372872 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab36b7-4e58-4f8e-b872-cf1c30b73802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:32.373350 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab36b7-4e58-4f8e-b872-cf1c30b73802 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:32.373775 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab36b7-4e58-4f8e-b872-cf1c30b73802 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:32.373889 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab36b7-4e58-4f8e-b872-cf1c30b73802 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:32.374589 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab36b7-4e58-4f8e-b872-cf1c30b73802 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:32.375109 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab36b7-4e58-4f8e-b872-cf1c30b73802 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:32.376330 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab36b7-4e58-4f8e-b872-cf1c30b73802 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:32.389646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 541/1102] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:20:32 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:20:32.395751 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab36b7-4e58-4f8e-b872-cf1c30b73802 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:32.452140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc82143-f819-4573-b307-6ad68095a8e7 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:32.471537 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:32.489681 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b6b1af-9d25-400f-91d9-261d70e8f0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:32.490359 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b6b1af-9d25-400f-91d9-261d70e8f0bc None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:32.490710 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b6b1af-9d25-400f-91d9-261d70e8f0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:32.490921 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b6b1af-9d25-400f-91d9-261d70e8f0bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:32.491733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b6b1af-9d25-400f-91d9-261d70e8f0bc None admin] RBAC: Adding query filter params (role.id=7342106a255b4bce8f150726602a8b95, scope.project.id=1e05b319c6e747a09159c629e77db830, user.id=b1bf26cbb3be4f749e7bf24ad624bb2d) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:32.492077 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b6b1af-9d25-400f-91d9-261d70e8f0bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:32.493595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b6b1af-9d25-400f-91d9-261d70e8f0bc None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:32.515982 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 562/1103] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:20:32 2022] GET /identity/v3/role_assignments?user.id=b1bf26cbb3be4f749e7bf24ad624bb2d&scope.project.id=1e05b319c6e747a09159c629e77db830&role.id=7342106a255b4bce8f150726602a8b95 => generated 610 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:32.518644 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab36b7-4e58-4f8e-b872-cf1c30b73802 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:32.566840 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202d679-7e48-4c16-a4fb-79fecec5d0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:32.567537 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202d679-7e48-4c16-a4fb-79fecec5d0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:32.567883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202d679-7e48-4c16-a4fb-79fecec5d0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:32.568333 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202d679-7e48-4c16-a4fb-79fecec5d0b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:32.577118 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202d679-7e48-4c16-a4fb-79fecec5d0b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:32.578841 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202d679-7e48-4c16-a4fb-79fecec5d0b6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:32.595678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-4202d679-7e48-4c16-a4fb-79fecec5d0b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:20:32.595678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:32.595678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:32.595678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:32.595678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:32.595678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:32.595678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:32.595678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:32.595678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:32.595678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:32.595678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:32.595678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:32.595678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:20:32.595678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:20:32.595678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:20:32.597212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:20:32.597212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:32.597212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:32.597212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:32.597212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:32.597212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:32.597212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:32.597212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:32.597212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:32.597212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:32.597212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:32.597212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:32.597212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:32.597212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:32.597212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:32.597212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:32.597212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:32.598915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:20:32.598915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:20:32.598915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:20:32.598915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:20:32.598915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:20:32.598915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:20:32.598915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:20:32.598915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:32.598915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 542/1104] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:20:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:32.606772 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b6b1af-9d25-400f-91d9-261d70e8f0bc None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:32.681430 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:32.749711 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b6b1af-9d25-400f-91d9-261d70e8f0bc None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:32.789890 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd0526-9098-4d5c-b36e-157b9cd7caf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:32.790548 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd0526-9098-4d5c-b36e-157b9cd7caf1 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:32.790828 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd0526-9098-4d5c-b36e-157b9cd7caf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:32.791103 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd0526-9098-4d5c-b36e-157b9cd7caf1 None admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:32.791884 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd0526-9098-4d5c-b36e-157b9cd7caf1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:32.792225 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd0526-9098-4d5c-b36e-157b9cd7caf1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:32.793737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd0526-9098-4d5c-b36e-157b9cd7caf1 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:32.808978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 563/1105] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:20:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:32.815800 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202d679-7e48-4c16-a4fb-79fecec5d0b6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:32.947057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202d679-7e48-4c16-a4fb-79fecec5d0b6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:32.980650 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:32.988906 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a131866-897a-41d6-9551-d179ce65cf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:32.989752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a131866-897a-41d6-9551-d179ce65cf16 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:32.990195 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a131866-897a-41d6-9551-d179ce65cf16 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:32.990560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a131866-897a-41d6-9551-d179ce65cf16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:32.991467 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a131866-897a-41d6-9551-d179ce65cf16 None admin] RBAC: Adding query filter params (role.id=9395de2ca6f34dfc85ea530fc0a14f45, scope.project.id=22768aba760b4307aab131d1312b2bf6, user.id=0d947aeb749546309a8d4b7938087ba0) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:32.991873 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a131866-897a-41d6-9551-d179ce65cf16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:32.993487 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a131866-897a-41d6-9551-d179ce65cf16 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:33.001667 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 543/1106] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:20:32 2022] GET /identity/v3/role_assignments?user.id=0d947aeb749546309a8d4b7938087ba0&scope.project.id=22768aba760b4307aab131d1312b2bf6&role.id=9395de2ca6f34dfc85ea530fc0a14f45 => generated 610 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:34.657512 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283342fd-0ba0-40a9-8f19-f6c634418403 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:34.658136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283342fd-0ba0-40a9-8f19-f6c634418403 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:34.658136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283342fd-0ba0-40a9-8f19-f6c634418403 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:34.659664 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 564/1107] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:34.666018 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5769fe5-45d8-4856-95f3-3b43e5112fdc None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:34.666644 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5769fe5-45d8-4856-95f3-3b43e5112fdc None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:34.666644 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5769fe5-45d8-4856-95f3-3b43e5112fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:34.685885 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b5769fe5-45d8-4856-95f3-3b43e5112fdc None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:34.687112 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b5769fe5-45d8-4856-95f3-3b43e5112fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:34.759990 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 544/1108] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:34 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 10:20:34.786054 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283342fd-0ba0-40a9-8f19-f6c634418403 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:34.902572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283342fd-0ba0-40a9-8f19-f6c634418403 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:34.921179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:34.942220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327e635-75a5-4253-873e-3608783e73a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:34.942932 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327e635-75a5-4253-873e-3608783e73a4 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:34.943207 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327e635-75a5-4253-873e-3608783e73a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:34.943460 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327e635-75a5-4253-873e-3608783e73a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:34.944749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327e635-75a5-4253-873e-3608783e73a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:34.946217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327e635-75a5-4253-873e-3608783e73a4 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:34.947879 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 565/1109] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:20:34 2022] GET /identity/v3/domains/default => generated 200 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:34.954424 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5769fe5-45d8-4856-95f3-3b43e5112fdc None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:35.074569 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5769fe5-45d8-4856-95f3-3b43e5112fdc None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:35.112669 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:35.124229 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f629cc-9a92-4649-abb1-8fb9bc42c1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:35.124861 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f629cc-9a92-4649-abb1-8fb9bc42c1e2 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:35.125141 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f629cc-9a92-4649-abb1-8fb9bc42c1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:35.125411 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f629cc-9a92-4649-abb1-8fb9bc42c1e2 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:35.126114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f629cc-9a92-4649-abb1-8fb9bc42c1e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:35.127568 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f629cc-9a92-4649-abb1-8fb9bc42c1e2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:35.150722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 545/1110] 10.222.0.121 () {60 vars in 1137 bytes} [Thu Aug 4 10:20:34 2022] POST /identity/v3/users => generated 287 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:20:37.327547 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249a130-d1e8-451e-897a-19f8f8d960af None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:37.328374 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249a130-d1e8-451e-897a-19f8f8d960af None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:37.328374 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249a130-d1e8-451e-897a-19f8f8d960af None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:37.329579 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 566/1111] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:37.336112 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864216c5-6e6b-4011-acb9-89de76cc8612 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:37.336809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864216c5-6e6b-4011-acb9-89de76cc8612 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:37.336809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864216c5-6e6b-4011-acb9-89de76cc8612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:37.358481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-864216c5-6e6b-4011-acb9-89de76cc8612 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:37.359751 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-864216c5-6e6b-4011-acb9-89de76cc8612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:37.441304 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 546/1112] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:37 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:37.466334 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249a130-d1e8-451e-897a-19f8f8d960af None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:37.575933 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249a130-d1e8-451e-897a-19f8f8d960af None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:37.613310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345abd7-9023-40b4-9a21-2a00edb9649e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:37.613987 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345abd7-9023-40b4-9a21-2a00edb9649e None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:37.614271 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345abd7-9023-40b4-9a21-2a00edb9649e None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:37.614551 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345abd7-9023-40b4-9a21-2a00edb9649e None admin] PATH_INFO: `/v3/roles/89b847fca3574b9083287279aee0f773` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:37.616133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345abd7-9023-40b4-9a21-2a00edb9649e None admin] RBAC: Authorizing `identity:get_role(role_id=89b847fca3574b9083287279aee0f773)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:37.617555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345abd7-9023-40b4-9a21-2a00edb9649e None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:37.619005 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 567/1113] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:20:37 2022] GET /identity/v3/roles/89b847fca3574b9083287279aee0f773 => generated 224 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:37.625397 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864216c5-6e6b-4011-acb9-89de76cc8612 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:37.737827 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864216c5-6e6b-4011-acb9-89de76cc8612 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:37.749995 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:37.776320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e4e3f-344f-45a8-81f5-48411feff716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:37.776988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e4e3f-344f-45a8-81f5-48411feff716 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:37.777273 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e4e3f-344f-45a8-81f5-48411feff716 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:37.777548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e4e3f-344f-45a8-81f5-48411feff716 None admin] PATH_INFO: `/v3/users/b28c29fafdde4d67a72cc616af5f16b5` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:37.793298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841e4e3f-344f-45a8-81f5-48411feff716 None admin] RBAC: Authorizing `identity:get_user(user_id=b28c29fafdde4d67a72cc616af5f16b5)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:37.794796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841e4e3f-344f-45a8-81f5-48411feff716 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:37.796621 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 547/1114] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:20:37 2022] GET /identity/v3/users/b28c29fafdde4d67a72cc616af5f16b5 => generated 287 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:37.805592 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345abd7-9023-40b4-9a21-2a00edb9649e None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:37.867657 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:37.917439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345abd7-9023-40b4-9a21-2a00edb9649e None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:37.956885 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfc18d-f2ae-4d29-963a-bd49e917db9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:37.957541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfc18d-f2ae-4d29-963a-bd49e917db9e None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:37.957831 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfc18d-f2ae-4d29-963a-bd49e917db9e None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:37.958114 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfc18d-f2ae-4d29-963a-bd49e917db9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:38.030948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cfc18d-f2ae-4d29-963a-bd49e917db9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:38.032479 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cfc18d-f2ae-4d29-963a-bd49e917db9e None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:38.091341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:38.147296 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 568/1115] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:20:37 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:20:38.153921 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e4e3f-344f-45a8-81f5-48411feff716 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:38.266028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e4e3f-344f-45a8-81f5-48411feff716 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:38.304546 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3c708-ae6e-4f23-b1ed-57cfde4a5f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:38.305211 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3c708-ae6e-4f23-b1ed-57cfde4a5f3e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:38.305495 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3c708-ae6e-4f23-b1ed-57cfde4a5f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:38.305772 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3c708-ae6e-4f23-b1ed-57cfde4a5f3e None admin] PATH_INFO: `/v3/projects/1e05b319c6e747a09159c629e77db830` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:38.307004 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3c708-ae6e-4f23-b1ed-57cfde4a5f3e None admin] RBAC: Authorizing `identity:get_project(project_id=1e05b319c6e747a09159c629e77db830)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:38.308531 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3c708-ae6e-4f23-b1ed-57cfde4a5f3e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:38.310269 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 548/1116] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:20:38 2022] GET /identity/v3/projects/1e05b319c6e747a09159c629e77db830 => generated 312 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:38.317047 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfc18d-f2ae-4d29-963a-bd49e917db9e None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:38.428786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfc18d-f2ae-4d29-963a-bd49e917db9e None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:38.465548 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc7636-adce-4b13-a894-a9b089a09849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:38.466194 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc7636-adce-4b13-a894-a9b089a09849 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:38.466477 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc7636-adce-4b13-a894-a9b089a09849 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:38.466754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc7636-adce-4b13-a894-a9b089a09849 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:38.467570 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc7636-adce-4b13-a894-a9b089a09849 None admin] RBAC: Adding query filter params (role.id=89b847fca3574b9083287279aee0f773, scope.project.id=1e05b319c6e747a09159c629e77db830, user.id=b28c29fafdde4d67a72cc616af5f16b5) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:38.467883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc7636-adce-4b13-a894-a9b089a09849 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:38.469417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc7636-adce-4b13-a894-a9b089a09849 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:38.476159 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 569/1117] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:20:38 2022] GET /identity/v3/role_assignments?user.id=b28c29fafdde4d67a72cc616af5f16b5&scope.project.id=1e05b319c6e747a09159c629e77db830&role.id=89b847fca3574b9083287279aee0f773 => generated 260 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:40.519982 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b21d34-c3eb-4a2f-bc7a-58a56e3797d2 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:40.520679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b21d34-c3eb-4a2f-bc7a-58a56e3797d2 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:40.520679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b21d34-c3eb-4a2f-bc7a-58a56e3797d2 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:40.521941 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 549/1118] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:40.528209 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c36cbd-0882-4793-96a9-2a53fd2a8e2c None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:40.528758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c36cbd-0882-4793-96a9-2a53fd2a8e2c None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:40.528758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c36cbd-0882-4793-96a9-2a53fd2a8e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:40.548675 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-a0c36cbd-0882-4793-96a9-2a53fd2a8e2c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:40.549913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-a0c36cbd-0882-4793-96a9-2a53fd2a8e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:40.562363 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:40.627746 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 570/1119] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:40 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:40.651694 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b21d34-c3eb-4a2f-bc7a-58a56e3797d2 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:40.690019 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:40.761540 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b21d34-c3eb-4a2f-bc7a-58a56e3797d2 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:40.797974 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c92fd-6d02-483e-9cf1-a560ebb97cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:40.798515 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c92fd-6d02-483e-9cf1-a560ebb97cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:40.798797 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c92fd-6d02-483e-9cf1-a560ebb97cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:40.799071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c92fd-6d02-483e-9cf1-a560ebb97cd3 None admin] PATH_INFO: `/v3/roles/89b847fca3574b9083287279aee0f773` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:40.800650 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c92fd-6d02-483e-9cf1-a560ebb97cd3 None admin] RBAC: Authorizing `identity:get_role(role_id=89b847fca3574b9083287279aee0f773)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:40.802088 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c92fd-6d02-483e-9cf1-a560ebb97cd3 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:40.803541 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 550/1120] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:20:40 2022] GET /identity/v3/roles/89b847fca3574b9083287279aee0f773 => generated 224 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:40.810084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c36cbd-0882-4793-96a9-2a53fd2a8e2c None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:40.920423 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c36cbd-0882-4793-96a9-2a53fd2a8e2c None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:40.963839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686eb45-8aef-4e1f-b551-fa9e9eaa9e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:40.965029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686eb45-8aef-4e1f-b551-fa9e9eaa9e6e None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:40.965528 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686eb45-8aef-4e1f-b551-fa9e9eaa9e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:40.965993 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686eb45-8aef-4e1f-b551-fa9e9eaa9e6e None admin] PATH_INFO: `/v3/users/b28c29fafdde4d67a72cc616af5f16b5` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:40.968130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686eb45-8aef-4e1f-b551-fa9e9eaa9e6e None admin] RBAC: Authorizing `identity:get_user(user_id=b28c29fafdde4d67a72cc616af5f16b5)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:40.970816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686eb45-8aef-4e1f-b551-fa9e9eaa9e6e None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:40.973938 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 571/1121] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:20:40 2022] GET /identity/v3/users/b28c29fafdde4d67a72cc616af5f16b5 => generated 287 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:40.981042 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c92fd-6d02-483e-9cf1-a560ebb97cd3 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:41.092251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c92fd-6d02-483e-9cf1-a560ebb97cd3 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:41.094958 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:41.129310 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce9281-b953-4d27-8420-855ed4cc329a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:41.129987 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce9281-b953-4d27-8420-855ed4cc329a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:41.130308 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce9281-b953-4d27-8420-855ed4cc329a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:41.130596 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce9281-b953-4d27-8420-855ed4cc329a None admin] PATH_INFO: `/v3/projects/1e05b319c6e747a09159c629e77db830` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:41.131809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce9281-b953-4d27-8420-855ed4cc329a None admin] RBAC: Authorizing `identity:get_project(project_id=1e05b319c6e747a09159c629e77db830)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:41.133344 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce9281-b953-4d27-8420-855ed4cc329a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:41.135086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 551/1122] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:20:40 2022] GET /identity/v3/projects/1e05b319c6e747a09159c629e77db830 => generated 312 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:41.142039 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686eb45-8aef-4e1f-b551-fa9e9eaa9e6e None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:41.162341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:41.268929 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686eb45-8aef-4e1f-b551-fa9e9eaa9e6e None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:41.317967 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414f96cc-5a61-4895-9b85-2906ad66625b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:41.318665 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f96cc-5a61-4895-9b85-2906ad66625b None admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:41.318947 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f96cc-5a61-4895-9b85-2906ad66625b None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:41.319224 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f96cc-5a61-4895-9b85-2906ad66625b None admin] PATH_INFO: `/v3/projects/1e05b319c6e747a09159c629e77db830/users/b28c29fafdde4d67a72cc616af5f16b5/roles/89b847fca3574b9083287279aee0f773` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:41.321525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414f96cc-5a61-4895-9b85-2906ad66625b None admin] RBAC: Authorizing `identity:create_grant(project_id=1e05b319c6e747a09159c629e77db830, user_id=b28c29fafdde4d67a72cc616af5f16b5, role_id=89b847fca3574b9083287279aee0f773)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:41.322997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414f96cc-5a61-4895-9b85-2906ad66625b None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:41.346551 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 572/1123] 10.222.0.121 () {58 vars in 1444 bytes} [Thu Aug 4 10:20:41 2022] PUT /identity/v3/projects/1e05b319c6e747a09159c629e77db830/users/b28c29fafdde4d67a72cc616af5f16b5/roles/89b847fca3574b9083287279aee0f773 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:20:43.409235 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd20a59-4ca1-4d8a-9b98-aeb1ef239183 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:43.409857 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd20a59-4ca1-4d8a-9b98-aeb1ef239183 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:43.409983 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd20a59-4ca1-4d8a-9b98-aeb1ef239183 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:43.411329 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 552/1124] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:43 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:43.417580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b46f91-dc59-4ecb-8425-e8fa875e01b6 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:43.418152 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b46f91-dc59-4ecb-8425-e8fa875e01b6 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:43.418152 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b46f91-dc59-4ecb-8425-e8fa875e01b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:43.438241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-a1b46f91-dc59-4ecb-8425-e8fa875e01b6 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:43.439475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-a1b46f91-dc59-4ecb-8425-e8fa875e01b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:43.510968 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 573/1125] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:43 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:43.535350 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd20a59-4ca1-4d8a-9b98-aeb1ef239183 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:43.660467 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd20a59-4ca1-4d8a-9b98-aeb1ef239183 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:43.696752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a7de2-cb1c-4061-aea0-52a2fd2c07ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:43.697441 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a7de2-cb1c-4061-aea0-52a2fd2c07ec None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:43.697725 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a7de2-cb1c-4061-aea0-52a2fd2c07ec None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:43.697994 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a7de2-cb1c-4061-aea0-52a2fd2c07ec None admin] PATH_INFO: `/v3/roles/89b847fca3574b9083287279aee0f773` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:43.699539 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a7de2-cb1c-4061-aea0-52a2fd2c07ec None admin] RBAC: Authorizing `identity:get_role(role_id=89b847fca3574b9083287279aee0f773)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:43.701067 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a7de2-cb1c-4061-aea0-52a2fd2c07ec None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:43.702559 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 553/1126] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:20:43 2022] GET /identity/v3/roles/89b847fca3574b9083287279aee0f773 => generated 224 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:43.709302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b46f91-dc59-4ecb-8425-e8fa875e01b6 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:43.713224 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:43.823969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b46f91-dc59-4ecb-8425-e8fa875e01b6 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:43.865912 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeefb94-506f-4edf-86b3-faba7f3b856c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:43.866625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeefb94-506f-4edf-86b3-faba7f3b856c None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:43.866918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeefb94-506f-4edf-86b3-faba7f3b856c None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:43.867205 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeefb94-506f-4edf-86b3-faba7f3b856c None admin] PATH_INFO: `/v3/users/b28c29fafdde4d67a72cc616af5f16b5` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:43.868509 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeefb94-506f-4edf-86b3-faba7f3b856c None admin] RBAC: Authorizing `identity:get_user(user_id=b28c29fafdde4d67a72cc616af5f16b5)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:43.870032 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeefb94-506f-4edf-86b3-faba7f3b856c None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:43.871795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 574/1127] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:20:43 2022] GET /identity/v3/users/b28c29fafdde4d67a72cc616af5f16b5 => generated 287 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:43.878423 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a7de2-cb1c-4061-aea0-52a2fd2c07ec None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:43.925808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:43.987841 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a7de2-cb1c-4061-aea0-52a2fd2c07ec None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:44.033391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab9997-c2ab-4b1d-9781-123edc9ff0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:44.034056 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab9997-c2ab-4b1d-9781-123edc9ff0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:44.034340 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab9997-c2ab-4b1d-9781-123edc9ff0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:44.034619 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab9997-c2ab-4b1d-9781-123edc9ff0b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:44.125481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab9997-c2ab-4b1d-9781-123edc9ff0b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:44.127033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab9997-c2ab-4b1d-9781-123edc9ff0b8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:44.171324 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:44.238391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 554/1128] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:20:43 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:20:44.245762 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeefb94-506f-4edf-86b3-faba7f3b856c None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:44.329102 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:44.378495 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeefb94-506f-4edf-86b3-faba7f3b856c None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:44.427375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b0366b-1615-4e42-b178-f79269148021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:44.428543 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0366b-1615-4e42-b178-f79269148021 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:44.428831 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0366b-1615-4e42-b178-f79269148021 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:44.429113 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0366b-1615-4e42-b178-f79269148021 None admin] PATH_INFO: `/v3/projects/1e05b319c6e747a09159c629e77db830` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:44.430624 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b0366b-1615-4e42-b178-f79269148021 None admin] RBAC: Authorizing `identity:get_project(project_id=1e05b319c6e747a09159c629e77db830)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:44.432333 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b0366b-1615-4e42-b178-f79269148021 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:44.434284 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 575/1129] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:20:44 2022] GET /identity/v3/projects/1e05b319c6e747a09159c629e77db830 => generated 312 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:44.441482 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab9997-c2ab-4b1d-9781-123edc9ff0b8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:44.566555 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab9997-c2ab-4b1d-9781-123edc9ff0b8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:44.606903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc4a99-49b7-4cbd-933b-dd7e6584e111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:44.607556 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc4a99-49b7-4cbd-933b-dd7e6584e111 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:44.607845 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc4a99-49b7-4cbd-933b-dd7e6584e111 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:44.608149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc4a99-49b7-4cbd-933b-dd7e6584e111 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:44.608990 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc4a99-49b7-4cbd-933b-dd7e6584e111 None admin] RBAC: Adding query filter params (role.id=89b847fca3574b9083287279aee0f773, scope.project.id=1e05b319c6e747a09159c629e77db830, user.id=b28c29fafdde4d67a72cc616af5f16b5) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:44.609309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc4a99-49b7-4cbd-933b-dd7e6584e111 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:44.610820 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc4a99-49b7-4cbd-933b-dd7e6584e111 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:44.618256 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 555/1130] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:20:44 2022] GET /identity/v3/role_assignments?user.id=b28c29fafdde4d67a72cc616af5f16b5&scope.project.id=1e05b319c6e747a09159c629e77db830&role.id=89b847fca3574b9083287279aee0f773 => generated 610 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:46.760991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c944e04-3488-4fa9-8143-aba2e6ed34ac None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:46.761746 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c944e04-3488-4fa9-8143-aba2e6ed34ac None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:46.761746 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c944e04-3488-4fa9-8143-aba2e6ed34ac None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:46.763084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 576/1131] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:46 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:46.769207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dd185-366f-4078-a12a-be3c0952502d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:46.769737 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dd185-366f-4078-a12a-be3c0952502d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:46.769737 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dd185-366f-4078-a12a-be3c0952502d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:46.777894 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:46.789967 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-507dd185-366f-4078-a12a-be3c0952502d None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:46.791246 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-507dd185-366f-4078-a12a-be3c0952502d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:46.865673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 556/1132] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:46 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:46.891524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c944e04-3488-4fa9-8143-aba2e6ed34ac None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:47.009813 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c944e04-3488-4fa9-8143-aba2e6ed34ac None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:47.048981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c913ccd-64e1-4960-8744-990a9cd3996a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:47.049669 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c913ccd-64e1-4960-8744-990a9cd3996a None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:47.049968 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c913ccd-64e1-4960-8744-990a9cd3996a None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:47.050239 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c913ccd-64e1-4960-8744-990a9cd3996a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:47.051614 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c913ccd-64e1-4960-8744-990a9cd3996a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:47.053207 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c913ccd-64e1-4960-8744-990a9cd3996a None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:47.054901 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 577/1133] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:20:46 2022] GET /identity/v3/domains/default => generated 200 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:47.062008 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507dd185-366f-4078-a12a-be3c0952502d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:47.176188 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507dd185-366f-4078-a12a-be3c0952502d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:47.216269 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a4be9-5ce7-46ee-be71-75f39635099f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:47.216933 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a4be9-5ce7-46ee-be71-75f39635099f None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:47.217220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a4be9-5ce7-46ee-be71-75f39635099f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:47.217491 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a4be9-5ce7-46ee-be71-75f39635099f None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:47.218248 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a4be9-5ce7-46ee-be71-75f39635099f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:47.219814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a4be9-5ce7-46ee-be71-75f39635099f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:47.225476 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:47.255430 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 557/1134] 10.222.0.121 () {60 vars in 1146 bytes} [Thu Aug 4 10:20:47 2022] POST /identity/v3/projects => generated 312 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:20:49.430010 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31023da7-d311-4b6b-beb3-dfd31dd85963 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:49.430994 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31023da7-d311-4b6b-beb3-dfd31dd85963 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:49.430994 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31023da7-d311-4b6b-beb3-dfd31dd85963 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:49.433905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 578/1135] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:49.438326 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669293d3-0ca2-488c-a24f-d8c00c0c0bfd None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:49.438902 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669293d3-0ca2-488c-a24f-d8c00c0c0bfd None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:49.438902 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669293d3-0ca2-488c-a24f-d8c00c0c0bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:49.460462 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-669293d3-0ca2-488c-a24f-d8c00c0c0bfd None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:49.461702 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-669293d3-0ca2-488c-a24f-d8c00c0c0bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:49.537721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 558/1136] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:49 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:49.564463 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31023da7-d311-4b6b-beb3-dfd31dd85963 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:49.581980 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:49.692057 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31023da7-d311-4b6b-beb3-dfd31dd85963 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:49.734643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210762b-b4f4-4e2c-b30c-63e2cb5b48d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:49.735299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210762b-b4f4-4e2c-b30c-63e2cb5b48d5 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:49.735597 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210762b-b4f4-4e2c-b30c-63e2cb5b48d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:49.735861 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210762b-b4f4-4e2c-b30c-63e2cb5b48d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:49.737491 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e210762b-b4f4-4e2c-b30c-63e2cb5b48d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:49.738978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e210762b-b4f4-4e2c-b30c-63e2cb5b48d5 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:49.740767 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 579/1137] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:20:49 2022] GET /identity/v3/domains/default => generated 200 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:49.747079 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669293d3-0ca2-488c-a24f-d8c00c0c0bfd None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:49.857368 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669293d3-0ca2-488c-a24f-d8c00c0c0bfd None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:49.875715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:49.901578 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11573ceb-4037-4f22-92fb-54c02b29cff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:49.902219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11573ceb-4037-4f22-92fb-54c02b29cff6 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:49.902500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11573ceb-4037-4f22-92fb-54c02b29cff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:49.902773 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11573ceb-4037-4f22-92fb-54c02b29cff6 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:49.903482 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11573ceb-4037-4f22-92fb-54c02b29cff6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:49.904980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11573ceb-4037-4f22-92fb-54c02b29cff6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:49.929396 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 559/1138] 10.222.0.121 () {60 vars in 1137 bytes} [Thu Aug 4 10:20:49 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:20:52.089920 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669f99c-be0e-4af3-a93a-2be7d4e1e067 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:52.090637 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669f99c-be0e-4af3-a93a-2be7d4e1e067 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:52.090637 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669f99c-be0e-4af3-a93a-2be7d4e1e067 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:52.091892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 580/1139] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:52.097856 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994340e3-f72f-47c4-bbeb-ca868ff31da5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:52.098417 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994340e3-f72f-47c4-bbeb-ca868ff31da5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:52.098417 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994340e3-f72f-47c4-bbeb-ca868ff31da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:52.120052 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-994340e3-f72f-47c4-bbeb-ca868ff31da5 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:52.121352 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-994340e3-f72f-47c4-bbeb-ca868ff31da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:52.206863 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 560/1140] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:52 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:52.231492 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669f99c-be0e-4af3-a93a-2be7d4e1e067 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:52.324483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:52.351489 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669f99c-be0e-4af3-a93a-2be7d4e1e067 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:52.389362 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb67a33-20b3-493f-9228-5e8c857855f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:52.390028 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb67a33-20b3-493f-9228-5e8c857855f4 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:52.390324 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb67a33-20b3-493f-9228-5e8c857855f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:52.390599 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb67a33-20b3-493f-9228-5e8c857855f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:52.401582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb67a33-20b3-493f-9228-5e8c857855f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:20:52.406634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-edb67a33-20b3-493f-9228-5e8c857855f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:20:52.406634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:52.406634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:52.406634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:52.406634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:52.406634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:52.406634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:52.406634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:52.406634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:52.406634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:52.406634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:52.406634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:52.406634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:20:52.406634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:20:52.406634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:20:52.406634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:20:52.408066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:20:52.408066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:20:52.408066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:20:52.408066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:20:52.408066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:52.408066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:52.408066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:52.408066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:52.408066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:52.408066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:52.408066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:52.408066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:52.408066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:52.408066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:52.408066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:52.408066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:52.409340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:52.409340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:52.409340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:52.409340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:52.409340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:20:52.409340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:20:52.409340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:20:52.409340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:20:52.409340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:20:52.409340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:20:52.409340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:20:52.409340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:52.409340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 581/1141] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:20:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:52.414743 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994340e3-f72f-47c4-bbeb-ca868ff31da5 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:52.531247 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994340e3-f72f-47c4-bbeb-ca868ff31da5 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:52.533304 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:52.574376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23ae85-ff30-4c21-9b0a-c39e6d913843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:52.575069 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23ae85-ff30-4c21-9b0a-c39e6d913843 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:52.575358 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23ae85-ff30-4c21-9b0a-c39e6d913843 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:52.575638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23ae85-ff30-4c21-9b0a-c39e6d913843 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:52.576436 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a23ae85-ff30-4c21-9b0a-c39e6d913843 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:52.576743 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a23ae85-ff30-4c21-9b0a-c39e6d913843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:52.578175 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a23ae85-ff30-4c21-9b0a-c39e6d913843 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:52.588661 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 561/1142] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:20:52 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:52.595162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb67a33-20b3-493f-9228-5e8c857855f4 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:52.718847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb67a33-20b3-493f-9228-5e8c857855f4 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:52.754992 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c94c9-be9d-4438-903d-f24d5e3a7241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:52.755693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c94c9-be9d-4438-903d-f24d5e3a7241 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:52.756011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c94c9-be9d-4438-903d-f24d5e3a7241 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:52.756287 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c94c9-be9d-4438-903d-f24d5e3a7241 None admin] PATH_INFO: `/v3/users/ffc7664eb5ef44a2966e78ed72f25d88` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:52.759326 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:52.772106 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c94c9-be9d-4438-903d-f24d5e3a7241 None admin] RBAC: Authorizing `identity:get_user(user_id=ffc7664eb5ef44a2966e78ed72f25d88)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:52.773647 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c94c9-be9d-4438-903d-f24d5e3a7241 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:52.775449 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 582/1143] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:20:52 2022] GET /identity/v3/users/ffc7664eb5ef44a2966e78ed72f25d88 => generated 287 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:52.784603 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23ae85-ff30-4c21-9b0a-c39e6d913843 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:52.905103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23ae85-ff30-4c21-9b0a-c39e6d913843 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:52.948873 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa75f24-72d8-4403-9acf-b70fd67fc280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:52.949538 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa75f24-72d8-4403-9acf-b70fd67fc280 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:52.949846 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa75f24-72d8-4403-9acf-b70fd67fc280 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:52.950118 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa75f24-72d8-4403-9acf-b70fd67fc280 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:52.994974 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:53.053947 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa75f24-72d8-4403-9acf-b70fd67fc280 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:53.056922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa75f24-72d8-4403-9acf-b70fd67fc280 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:53.171461 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 562/1144] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:20:52 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:20:53.178594 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c94c9-be9d-4438-903d-f24d5e3a7241 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:53.297664 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c94c9-be9d-4438-903d-f24d5e3a7241 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:53.334745 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b69b106-e621-47c3-882b-ab7bfc17b856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:53.335414 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69b106-e621-47c3-882b-ab7bfc17b856 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:53.335685 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69b106-e621-47c3-882b-ab7bfc17b856 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:53.335991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69b106-e621-47c3-882b-ab7bfc17b856 None admin] PATH_INFO: `/v3/projects/344017a860384b18ba65077208f7244f` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:53.337299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b69b106-e621-47c3-882b-ab7bfc17b856 None admin] RBAC: Authorizing `identity:get_project(project_id=344017a860384b18ba65077208f7244f)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:53.338784 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b69b106-e621-47c3-882b-ab7bfc17b856 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:53.340583 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 583/1145] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:20:53 2022] GET /identity/v3/projects/344017a860384b18ba65077208f7244f => generated 312 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:53.348403 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa75f24-72d8-4403-9acf-b70fd67fc280 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:53.418647 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:53.459915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa75f24-72d8-4403-9acf-b70fd67fc280 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:53.495671 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de980da3-8fb0-4cb2-b900-2e788b899553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:53.496320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de980da3-8fb0-4cb2-b900-2e788b899553 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:53.496599 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de980da3-8fb0-4cb2-b900-2e788b899553 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:53.496871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de980da3-8fb0-4cb2-b900-2e788b899553 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:53.497664 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de980da3-8fb0-4cb2-b900-2e788b899553 None admin] RBAC: Adding query filter params (role.id=7342106a255b4bce8f150726602a8b95, scope.project.id=344017a860384b18ba65077208f7244f, user.id=ffc7664eb5ef44a2966e78ed72f25d88) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:53.497970 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de980da3-8fb0-4cb2-b900-2e788b899553 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:53.499605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de980da3-8fb0-4cb2-b900-2e788b899553 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:53.506164 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 563/1146] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:20:53 2022] GET /identity/v3/role_assignments?user.id=ffc7664eb5ef44a2966e78ed72f25d88&scope.project.id=344017a860384b18ba65077208f7244f&role.id=7342106a255b4bce8f150726602a8b95 => generated 260 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:55.560170 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4445a5a-99f6-4e18-bddb-e1f7a10702f2 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:55.561286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4445a5a-99f6-4e18-bddb-e1f7a10702f2 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:55.561650 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4445a5a-99f6-4e18-bddb-e1f7a10702f2 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:55.563134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 584/1147] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:55 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:55.571744 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696091c5-21f2-4775-abe5-779fab4421fb None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:55.572515 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696091c5-21f2-4775-abe5-779fab4421fb None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:55.572515 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696091c5-21f2-4775-abe5-779fab4421fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:55.595130 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-696091c5-21f2-4775-abe5-779fab4421fb None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:55.596428 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-696091c5-21f2-4775-abe5-779fab4421fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:55.684184 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 564/1148] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:55 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:55.723191 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4445a5a-99f6-4e18-bddb-e1f7a10702f2 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:55.817243 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:55.870948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4445a5a-99f6-4e18-bddb-e1f7a10702f2 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:55.913178 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b1b37-95c9-47a1-9e61-f84fdc173d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:55.913882 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b1b37-95c9-47a1-9e61-f84fdc173d34 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:55.914168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b1b37-95c9-47a1-9e61-f84fdc173d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:55.914449 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b1b37-95c9-47a1-9e61-f84fdc173d34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:55.928315 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b1b37-95c9-47a1-9e61-f84fdc173d34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:20:55.933738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-a77b1b37-95c9-47a1-9e61-f84fdc173d34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:20:55.933738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:55.933738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:55.933738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:55.933738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:55.933738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:55.933738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:55.933738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:55.933738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:55.933738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:55.933738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:55.933738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:55.933738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:20:55.933738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:20:55.933738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:20:55.933738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:20:55.934976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:20:55.934976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:20:55.934976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:20:55.934976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:20:55.934976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:55.934976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:55.934976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:55.934976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:55.934976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:55.934976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:55.934976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:55.934976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:55.934976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:55.934976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:55.934976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:55.934976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:55.936086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:55.936086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:55.936086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:55.936086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:55.936086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:20:55.936086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:20:55.936086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:20:55.936086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:20:55.936086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:20:55.936086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:20:55.936086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:20:55.936086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:20:55.936086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 585/1149] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:20:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:55.944333 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696091c5-21f2-4775-abe5-779fab4421fb None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:56.023819 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:56.070470 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696091c5-21f2-4775-abe5-779fab4421fb None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:56.108051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9183a-9f7c-4966-b01a-4a4acc57b4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:56.108696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9183a-9f7c-4966-b01a-4a4acc57b4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:56.108980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9183a-9f7c-4966-b01a-4a4acc57b4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:56.109252 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9183a-9f7c-4966-b01a-4a4acc57b4b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:56.109991 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9183a-9f7c-4966-b01a-4a4acc57b4b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:56.110295 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9183a-9f7c-4966-b01a-4a4acc57b4b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:56.111702 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9183a-9f7c-4966-b01a-4a4acc57b4b2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:56.124394 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 565/1150] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:20:55 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:56.134248 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b1b37-95c9-47a1-9e61-f84fdc173d34 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:56.245731 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b1b37-95c9-47a1-9e61-f84fdc173d34 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:56.270180 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:56.282925 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778fb9ff-1a13-4d98-8a53-09876ca8b0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:56.283606 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778fb9ff-1a13-4d98-8a53-09876ca8b0c5 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:56.283889 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778fb9ff-1a13-4d98-8a53-09876ca8b0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:56.284203 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778fb9ff-1a13-4d98-8a53-09876ca8b0c5 None admin] PATH_INFO: `/v3/users/ffc7664eb5ef44a2966e78ed72f25d88` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:56.285459 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778fb9ff-1a13-4d98-8a53-09876ca8b0c5 None admin] RBAC: Authorizing `identity:get_user(user_id=ffc7664eb5ef44a2966e78ed72f25d88)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:56.287020 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778fb9ff-1a13-4d98-8a53-09876ca8b0c5 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:56.288761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 586/1151] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:20:56 2022] GET /identity/v3/users/ffc7664eb5ef44a2966e78ed72f25d88 => generated 287 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:56.295183 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9183a-9f7c-4966-b01a-4a4acc57b4b2 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:56.410341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9183a-9f7c-4966-b01a-4a4acc57b4b2 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:56.432339 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:56.447575 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edff2b5-8cfa-4879-b1f6-20e703311ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:56.448282 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edff2b5-8cfa-4879-b1f6-20e703311ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:56.448572 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edff2b5-8cfa-4879-b1f6-20e703311ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:56.448853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edff2b5-8cfa-4879-b1f6-20e703311ec0 None admin] PATH_INFO: `/v3/projects/344017a860384b18ba65077208f7244f` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:56.450057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edff2b5-8cfa-4879-b1f6-20e703311ec0 None admin] RBAC: Authorizing `identity:get_project(project_id=344017a860384b18ba65077208f7244f)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:56.451527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edff2b5-8cfa-4879-b1f6-20e703311ec0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:56.453304 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 566/1152] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:20:56 2022] GET /identity/v3/projects/344017a860384b18ba65077208f7244f => generated 312 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:56.460395 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778fb9ff-1a13-4d98-8a53-09876ca8b0c5 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:56.570631 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778fb9ff-1a13-4d98-8a53-09876ca8b0c5 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:56.608669 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21937518-e986-4519-8b3e-3cebcbec965f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:56.609362 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21937518-e986-4519-8b3e-3cebcbec965f None admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:56.609649 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21937518-e986-4519-8b3e-3cebcbec965f None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:56.610103 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21937518-e986-4519-8b3e-3cebcbec965f None admin] PATH_INFO: `/v3/projects/344017a860384b18ba65077208f7244f/users/ffc7664eb5ef44a2966e78ed72f25d88/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:56.612616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21937518-e986-4519-8b3e-3cebcbec965f None admin] RBAC: Authorizing `identity:create_grant(project_id=344017a860384b18ba65077208f7244f, user_id=ffc7664eb5ef44a2966e78ed72f25d88, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:56.614120 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21937518-e986-4519-8b3e-3cebcbec965f None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:56.630857 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 587/1153] 10.222.0.121 () {58 vars in 1444 bytes} [Thu Aug 4 10:20:56 2022] PUT /identity/v3/projects/344017a860384b18ba65077208f7244f/users/ffc7664eb5ef44a2966e78ed72f25d88/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:20:58.812686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a6713-bc76-45e1-bbdd-750866aa6797 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:58.813418 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a6713-bc76-45e1-bbdd-750866aa6797 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:58.813418 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a6713-bc76-45e1-bbdd-750866aa6797 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:58.814693 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 567/1154] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:20:58 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:20:58.821233 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c5a97-e950-4738-b22a-5cb877dd6e8f None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:58.821727 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c5a97-e950-4738-b22a-5cb877dd6e8f None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:58.821823 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c5a97-e950-4738-b22a-5cb877dd6e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:58.852439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-937c5a97-e950-4738-b22a-5cb877dd6e8f None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:20:58.854513 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-937c5a97-e950-4738-b22a-5cb877dd6e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:20:58.908922 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:58.949112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 588/1155] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:20:58 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:20:58.995990 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456a6713-bc76-45e1-bbdd-750866aa6797 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:59.112617 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456a6713-bc76-45e1-bbdd-750866aa6797 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:59.148231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6afe5-3194-44fa-91bc-375984497264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:59.148886 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6afe5-3194-44fa-91bc-375984497264 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:59.149176 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6afe5-3194-44fa-91bc-375984497264 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:59.149458 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6afe5-3194-44fa-91bc-375984497264 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:59.161434 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc6afe5-3194-44fa-91bc-375984497264 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:20:59.166322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-7fc6afe5-3194-44fa-91bc-375984497264 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:20:59.166322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:20:59.166322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:20:59.166322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:20:59.166322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:20:59.166322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:20:59.166322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:20:59.166322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:20:59.166322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:20:59.166322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:20:59.166322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:20:59.166322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:20:59.166322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:20:59.166322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:20:59.166322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:20:59.166322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:20:59.167510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:20:59.167510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:20:59.167510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:20:59.167510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:20:59.167510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:20:59.167510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:20:59.167510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:20:59.167510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:20:59.167510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:20:59.167510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:20:59.167510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:20:59.167510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:20:59.167510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:20:59.167510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:20:59.167510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:20:59.167510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:20:59.168728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:20:59.168728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:20:59.168728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:20:59.168728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:20:59.168728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:20:59.168728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:20:59.168728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:20:59.168728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:20:59.168728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:20:59.168728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:20:59.168728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:20:59.168728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:20:59.168728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 568/1156] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:20:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:20:59.174585 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c5a97-e950-4738-b22a-5cb877dd6e8f None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:59.291687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c5a97-e950-4738-b22a-5cb877dd6e8f None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:59.338645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc964a-9a83-4720-920d-2186abf79ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:59.339286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc964a-9a83-4720-920d-2186abf79ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:59.339570 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc964a-9a83-4720-920d-2186abf79ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:59.339848 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc964a-9a83-4720-920d-2186abf79ac0 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:59.340892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc964a-9a83-4720-920d-2186abf79ac0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:20:59.340892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc964a-9a83-4720-920d-2186abf79ac0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:59.342469 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc964a-9a83-4720-920d-2186abf79ac0 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:59.352329 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 589/1157] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:20:59 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:59.359184 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6afe5-3194-44fa-91bc-375984497264 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:59.413548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:59.474186 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6afe5-3194-44fa-91bc-375984497264 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:59.535736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424997a-45ce-434d-aa82-5a08ab16ada5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:59.536491 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424997a-45ce-434d-aa82-5a08ab16ada5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:59.536764 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424997a-45ce-434d-aa82-5a08ab16ada5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:59.537045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424997a-45ce-434d-aa82-5a08ab16ada5 None admin] PATH_INFO: `/v3/users/ffc7664eb5ef44a2966e78ed72f25d88` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:59.538378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424997a-45ce-434d-aa82-5a08ab16ada5 None admin] RBAC: Authorizing `identity:get_user(user_id=ffc7664eb5ef44a2966e78ed72f25d88)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:59.540080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424997a-45ce-434d-aa82-5a08ab16ada5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:59.541739 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 569/1158] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:20:59 2022] GET /identity/v3/users/ffc7664eb5ef44a2966e78ed72f25d88 => generated 287 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:20:59.548568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc964a-9a83-4720-920d-2186abf79ac0 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:20:59.596431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:59.657706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc964a-9a83-4720-920d-2186abf79ac0 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:20:59.696079 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54a799-087d-41fe-8b87-0dce2b343e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:20:59.696620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54a799-087d-41fe-8b87-0dce2b343e21 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:20:59.696913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54a799-087d-41fe-8b87-0dce2b343e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:20:59.697190 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54a799-087d-41fe-8b87-0dce2b343e21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:20:59.776167 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54a799-087d-41fe-8b87-0dce2b343e21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:20:59.778175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54a799-087d-41fe-8b87-0dce2b343e21 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:20:59.826598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:20:59.895577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 590/1159] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:20:59 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:20:59.902353 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424997a-45ce-434d-aa82-5a08ab16ada5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:00.013997 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424997a-45ce-434d-aa82-5a08ab16ada5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:00.031232 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:00.051129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62219e29-838d-4937-b9d0-d545c15fa6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:00.051871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62219e29-838d-4937-b9d0-d545c15fa6a3 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:00.052195 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62219e29-838d-4937-b9d0-d545c15fa6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:00.052477 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62219e29-838d-4937-b9d0-d545c15fa6a3 None admin] PATH_INFO: `/v3/projects/344017a860384b18ba65077208f7244f` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:00.053836 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62219e29-838d-4937-b9d0-d545c15fa6a3 None admin] RBAC: Authorizing `identity:get_project(project_id=344017a860384b18ba65077208f7244f)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:00.055333 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62219e29-838d-4937-b9d0-d545c15fa6a3 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:00.057097 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 570/1160] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:20:59 2022] GET /identity/v3/projects/344017a860384b18ba65077208f7244f => generated 312 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:21:00.064654 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54a799-087d-41fe-8b87-0dce2b343e21 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:00.177392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54a799-087d-41fe-8b87-0dce2b343e21 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:00.216928 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca588e5-d94b-480d-87e4-4e553eb4bdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:00.217565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca588e5-d94b-480d-87e4-4e553eb4bdd2 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:00.217849 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca588e5-d94b-480d-87e4-4e553eb4bdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:00.218124 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca588e5-d94b-480d-87e4-4e553eb4bdd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:00.218935 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca588e5-d94b-480d-87e4-4e553eb4bdd2 None admin] RBAC: Adding query filter params (role.id=7342106a255b4bce8f150726602a8b95, scope.project.id=344017a860384b18ba65077208f7244f, user.id=ffc7664eb5ef44a2966e78ed72f25d88) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:21:00.219266 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca588e5-d94b-480d-87e4-4e553eb4bdd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:00.220861 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca588e5-d94b-480d-87e4-4e553eb4bdd2 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:00.227302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 591/1161] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:21:00 2022] GET /identity/v3/role_assignments?user.id=ffc7664eb5ef44a2966e78ed72f25d88&scope.project.id=344017a860384b18ba65077208f7244f&role.id=7342106a255b4bce8f150726602a8b95 => generated 610 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:21:02.339570 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb93e1-6010-457f-a113-c47cc3096b8e None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:02.340299 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb93e1-6010-457f-a113-c47cc3096b8e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:02.340299 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb93e1-6010-457f-a113-c47cc3096b8e None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:02.341719 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 571/1162] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:21:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:21:02.347238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098407e7-9aaf-455d-817f-6361d8cbb9c7 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:02.347721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098407e7-9aaf-455d-817f-6361d8cbb9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:02.347815 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098407e7-9aaf-455d-817f-6361d8cbb9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:02.361891 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:02.368215 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-098407e7-9aaf-455d-817f-6361d8cbb9c7 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:21:02.369445 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-098407e7-9aaf-455d-817f-6361d8cbb9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:21:02.440436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 592/1163] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:21:02 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:21:02.463572 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb93e1-6010-457f-a113-c47cc3096b8e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:02.579270 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb93e1-6010-457f-a113-c47cc3096b8e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:02.615630 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f13b06-108c-44f0-8202-32fd2859c924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:02.616450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f13b06-108c-44f0-8202-32fd2859c924 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:02.616793 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f13b06-108c-44f0-8202-32fd2859c924 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:02.617186 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f13b06-108c-44f0-8202-32fd2859c924 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:02.673466 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:02.690347 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f13b06-108c-44f0-8202-32fd2859c924 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:02.691845 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f13b06-108c-44f0-8202-32fd2859c924 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:02.794635 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 572/1164] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:21:02 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 10:21:02.801198 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098407e7-9aaf-455d-817f-6361d8cbb9c7 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:02.909737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098407e7-9aaf-455d-817f-6361d8cbb9c7 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:02.945363 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b3b92-2836-436f-a820-95e35874e306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:02.946027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b3b92-2836-436f-a820-95e35874e306 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:02.946316 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b3b92-2836-436f-a820-95e35874e306 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:02.946598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b3b92-2836-436f-a820-95e35874e306 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:02.949648 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:02.954706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b3b92-2836-436f-a820-95e35874e306 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:02.956336 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b3b92-2836-436f-a820-95e35874e306 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:02.968145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-0b6b3b92-2836-436f-a820-95e35874e306 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 04 10:21:02.968145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:21:02.968145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 04 10:21:02.968145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 04 10:21:02.968145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:21:02.968145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:21:02.968145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:21:02.968145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:21:02.968145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 04 10:21:02.968145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:21:02.968145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:21:02.968145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:21:02.968145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:21:02.968145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:21:02.968145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:21:02.968145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:21:02.968145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:21:02.970090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:21:02.970090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:21:02.970090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:21:02.970090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:21:02.970090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:21:02.970090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:21:02.970090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:21:02.970090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:21:02.970090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:21:02.970090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:21:02.970090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:21:02.970090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:21:02.970090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:21:02.970090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:21:02.970090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:21:02.970090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:21:02.971900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:21:02.971900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:21:02.971900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:21:02.971900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:21:02.971900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:21:02.971900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:21:02.971900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:21:02.971900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:21:02.971900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:21:02.971900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:21:02.971900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 04 10:21:02.971900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:21:02.971900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 04 10:21:02.971900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:21:02.971900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 593/1165] 10.222.0.121 () {56 vars in 1122 bytes} [Thu Aug 4 10:21:02 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:21:02.976270 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f13b06-108c-44f0-8202-32fd2859c924 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:03.065875 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:03.085008 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f13b06-108c-44f0-8202-32fd2859c924 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:03.121186 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a451b-d8ce-4eee-80da-443c6d74c251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:03.121818 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a451b-d8ce-4eee-80da-443c6d74c251 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:03.122103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a451b-d8ce-4eee-80da-443c6d74c251 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:03.122376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a451b-d8ce-4eee-80da-443c6d74c251 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:03.123080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a451b-d8ce-4eee-80da-443c6d74c251 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:21:03.123387 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a451b-d8ce-4eee-80da-443c6d74c251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:03.124814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a451b-d8ce-4eee-80da-443c6d74c251 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:03.132882 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 573/1166] 10.222.0.121 () {56 vars in 1120 bytes} [Thu Aug 4 10:21:02 2022] GET /identity/v3/domains?name=swift_test => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:21:05.201154 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e82dcc-ee9b-48be-aa06-c279af39b642 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:05.201780 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e82dcc-ee9b-48be-aa06-c279af39b642 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:05.201780 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e82dcc-ee9b-48be-aa06-c279af39b642 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:05.203063 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 594/1167] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:21:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:21:05.212481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84539b1c-520e-4286-a03c-4566ac25edf6 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:05.212481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84539b1c-520e-4286-a03c-4566ac25edf6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:05.212481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84539b1c-520e-4286-a03c-4566ac25edf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:05.232119 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-84539b1c-520e-4286-a03c-4566ac25edf6 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:21:05.233392 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-84539b1c-520e-4286-a03c-4566ac25edf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:21:05.313954 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 574/1168] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:21:05 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:21:05.338374 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e82dcc-ee9b-48be-aa06-c279af39b642 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:05.470222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e82dcc-ee9b-48be-aa06-c279af39b642 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:05.514695 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a090986-684d-4b79-bc56-0e548e759612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:05.515367 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a090986-684d-4b79-bc56-0e548e759612 None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:05.515655 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a090986-684d-4b79-bc56-0e548e759612 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:05.515935 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a090986-684d-4b79-bc56-0e548e759612 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:05.516758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a090986-684d-4b79-bc56-0e548e759612 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:05.518205 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a090986-684d-4b79-bc56-0e548e759612 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:05.539619 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 595/1169] 10.222.0.121 () {60 vars in 1143 bytes} [Thu Aug 4 10:21:05 2022] POST /identity/v3/domains => generated 268 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:21:07.697634 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9c6ef-3e14-4cde-95ea-e284bec9c2f2 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:07.698304 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9c6ef-3e14-4cde-95ea-e284bec9c2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:07.698304 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9c6ef-3e14-4cde-95ea-e284bec9c2f2 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:07.699600 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 575/1170] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:21:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:21:07.706073 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3797a19-cd53-459e-ac54-37d79b0b9a41 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:07.706601 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3797a19-cd53-459e-ac54-37d79b0b9a41 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:07.706716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3797a19-cd53-459e-ac54-37d79b0b9a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:07.725948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-d3797a19-cd53-459e-ac54-37d79b0b9a41 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:21:07.727179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d3797a19-cd53-459e-ac54-37d79b0b9a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:21:07.790289 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:07.813841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 596/1171] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:21:07 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:21:07.845857 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9c6ef-3e14-4cde-95ea-e284bec9c2f2 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:07.935475 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:07.961879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9c6ef-3e14-4cde-95ea-e284bec9c2f2 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:08.005748 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441020ed-4c5d-42e2-994f-388bc59ac7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:08.006401 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441020ed-4c5d-42e2-994f-388bc59ac7d5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:08.006668 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441020ed-4c5d-42e2-994f-388bc59ac7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:08.006963 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441020ed-4c5d-42e2-994f-388bc59ac7d5 None admin] PATH_INFO: `/v3/domains/8c351d286014475ab0f0e7643a34dbfa` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:08.021383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441020ed-4c5d-42e2-994f-388bc59ac7d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8c351d286014475ab0f0e7643a34dbfa)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:08.022902 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441020ed-4c5d-42e2-994f-388bc59ac7d5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:08.024807 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 576/1172] 10.222.0.121 () {56 vars in 1188 bytes} [Thu Aug 4 10:21:07 2022] GET /identity/v3/domains/8c351d286014475ab0f0e7643a34dbfa => generated 268 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:21:08.035201 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3797a19-cd53-459e-ac54-37d79b0b9a41 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:08.147457 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3797a19-cd53-459e-ac54-37d79b0b9a41 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:08.189105 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e20d71-9554-497a-81c8-4365f6787bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:08.189778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e20d71-9554-497a-81c8-4365f6787bef None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:08.190068 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e20d71-9554-497a-81c8-4365f6787bef None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:08.190341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e20d71-9554-497a-81c8-4365f6787bef None admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:08.191082 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e20d71-9554-497a-81c8-4365f6787bef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:08.192738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e20d71-9554-497a-81c8-4365f6787bef None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:08.225801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 597/1173] 10.222.0.121 () {60 vars in 1146 bytes} [Thu Aug 4 10:21:08 2022] POST /identity/v3/projects => generated 362 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:21:10.322971 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af499d3-a350-419e-bcc0-bec77937082e None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:10.323616 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af499d3-a350-419e-bcc0-bec77937082e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:10.323616 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af499d3-a350-419e-bcc0-bec77937082e None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:10.324929 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 577/1174] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:21:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:21:10.330471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb576bac-a1f4-4e98-944d-7b93cc3fab3a None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:10.330950 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb576bac-a1f4-4e98-944d-7b93cc3fab3a None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:10.331071 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb576bac-a1f4-4e98-944d-7b93cc3fab3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:10.350181 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-fb576bac-a1f4-4e98-944d-7b93cc3fab3a None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:21:10.351394 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-fb576bac-a1f4-4e98-944d-7b93cc3fab3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:21:10.382975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:10.426050 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 598/1175] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:21:10 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:21:10.452115 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af499d3-a350-419e-bcc0-bec77937082e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:10.577402 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af499d3-a350-419e-bcc0-bec77937082e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:10.614487 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a3655-f6a9-4416-88ef-83c3e58a71b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:10.615028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a3655-f6a9-4416-88ef-83c3e58a71b1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:10.615308 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a3655-f6a9-4416-88ef-83c3e58a71b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:10.615590 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a3655-f6a9-4416-88ef-83c3e58a71b1 None admin] PATH_INFO: `/v3/domains/8c351d286014475ab0f0e7643a34dbfa` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:10.616832 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a3655-f6a9-4416-88ef-83c3e58a71b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8c351d286014475ab0f0e7643a34dbfa)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:10.618299 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a3655-f6a9-4416-88ef-83c3e58a71b1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:10.619971 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 578/1176] 10.222.0.121 () {56 vars in 1188 bytes} [Thu Aug 4 10:21:10 2022] GET /identity/v3/domains/8c351d286014475ab0f0e7643a34dbfa => generated 268 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:21:10.629345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb576bac-a1f4-4e98-944d-7b93cc3fab3a None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:10.744942 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb576bac-a1f4-4e98-944d-7b93cc3fab3a None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:10.783036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a631f8-5b6a-457c-a599-9b8fe7bb051d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:10.783676 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a631f8-5b6a-457c-a599-9b8fe7bb051d None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:10.783987 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a631f8-5b6a-457c-a599-9b8fe7bb051d None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:10.784266 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a631f8-5b6a-457c-a599-9b8fe7bb051d None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:10.784997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a631f8-5b6a-457c-a599-9b8fe7bb051d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:10.786525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a631f8-5b6a-457c-a599-9b8fe7bb051d None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:10.816414 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 599/1177] 10.222.0.121 () {60 vars in 1137 bytes} [Thu Aug 4 10:21:10 2022] POST /identity/v3/users => generated 312 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:21:12.935608 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97122940-9cca-418a-993b-a08a31f21a45 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:12.936245 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97122940-9cca-418a-993b-a08a31f21a45 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:12.936245 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97122940-9cca-418a-993b-a08a31f21a45 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:12.937470 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 579/1178] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:21:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:21:12.943295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa056a2-8f7b-493e-900b-a05f12c6307c None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:12.943845 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa056a2-8f7b-493e-900b-a05f12c6307c None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:12.943845 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa056a2-8f7b-493e-900b-a05f12c6307c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:12.963543 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-daa056a2-8f7b-493e-900b-a05f12c6307c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:21:12.964879 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-daa056a2-8f7b-493e-900b-a05f12c6307c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:21:12.983651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:13.036469 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 600/1179] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:21:12 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:21:13.061197 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97122940-9cca-418a-993b-a08a31f21a45 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:13.073383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:13.183220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97122940-9cca-418a-993b-a08a31f21a45 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:13.216600 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163315ec-8232-45a4-be96-84448daa4a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:13.217185 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163315ec-8232-45a4-be96-84448daa4a6d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:13.217464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163315ec-8232-45a4-be96-84448daa4a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:13.217737 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163315ec-8232-45a4-be96-84448daa4a6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:13.229811 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163315ec-8232-45a4-be96-84448daa4a6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:21:13.234888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-163315ec-8232-45a4-be96-84448daa4a6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:21:13.234888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:21:13.234888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:21:13.234888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:21:13.234888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:21:13.234888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:21:13.234888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:21:13.234888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:21:13.234888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:21:13.234888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:21:13.234888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:21:13.234888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:21:13.234888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:21:13.234888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:21:13.234888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:21:13.234888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:21:13.236058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:21:13.236058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:21:13.236058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:21:13.236058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:21:13.236058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:21:13.236058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:21:13.236058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:21:13.236058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:21:13.236058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:21:13.236058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:21:13.236058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:21:13.236058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:21:13.236058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:21:13.236058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:21:13.236058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:21:13.236058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:21:13.237461 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:21:13.237461 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:21:13.237461 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:21:13.237461 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:21:13.237461 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:21:13.237461 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:21:13.237461 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:21:13.237461 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:21:13.237461 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:21:13.237461 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:21:13.237461 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:21:13.237461 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:21:13.237461 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 580/1180] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:21:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:21:13.243306 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa056a2-8f7b-493e-900b-a05f12c6307c None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:13.347554 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa056a2-8f7b-493e-900b-a05f12c6307c None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:13.384724 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9449f-e084-4d36-8135-ce499a004139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:13.385358 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9449f-e084-4d36-8135-ce499a004139 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:13.385644 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9449f-e084-4d36-8135-ce499a004139 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:13.385900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9449f-e084-4d36-8135-ce499a004139 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:13.386667 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c9449f-e084-4d36-8135-ce499a004139 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:21:13.386979 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c9449f-e084-4d36-8135-ce499a004139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:13.388412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c9449f-e084-4d36-8135-ce499a004139 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:13.397170 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 601/1181] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:21:13 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:21:13.403596 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163315ec-8232-45a4-be96-84448daa4a6d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:13.490458 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:13.529679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163315ec-8232-45a4-be96-84448daa4a6d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:13.565364 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b98c1c-5b54-4686-9170-1cfd607f0089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:13.566029 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b98c1c-5b54-4686-9170-1cfd607f0089 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:13.566287 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b98c1c-5b54-4686-9170-1cfd607f0089 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:13.566585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b98c1c-5b54-4686-9170-1cfd607f0089 None admin] PATH_INFO: `/v3/users/a3a63d4496df45f9802de94f7306505b` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:13.580645 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b98c1c-5b54-4686-9170-1cfd607f0089 None admin] RBAC: Authorizing `identity:get_user(user_id=a3a63d4496df45f9802de94f7306505b)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:13.582169 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b98c1c-5b54-4686-9170-1cfd607f0089 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:13.583931 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 581/1182] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:21:13 2022] GET /identity/v3/users/a3a63d4496df45f9802de94f7306505b => generated 312 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:21:13.590201 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9449f-e084-4d36-8135-ce499a004139 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:13.611491 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:13.700615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9449f-e084-4d36-8135-ce499a004139 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:13.736384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da9a71-33c9-4db2-9358-880c21c2d0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:13.737368 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da9a71-33c9-4db2-9358-880c21c2d0da None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:13.737662 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da9a71-33c9-4db2-9358-880c21c2d0da None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:13.737939 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da9a71-33c9-4db2-9358-880c21c2d0da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:13.813267 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da9a71-33c9-4db2-9358-880c21c2d0da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:13.814811 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da9a71-33c9-4db2-9358-880c21c2d0da None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:13.839749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:13.940902 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 602/1183] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:21:13 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:21:13.948537 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b98c1c-5b54-4686-9170-1cfd607f0089 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:14.056699 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b98c1c-5b54-4686-9170-1cfd607f0089 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:14.080745 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:14.092235 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59178b1-f1ac-4732-9a3a-e6ae9f8230d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:14.092901 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59178b1-f1ac-4732-9a3a-e6ae9f8230d1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:14.093185 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59178b1-f1ac-4732-9a3a-e6ae9f8230d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:14.093458 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59178b1-f1ac-4732-9a3a-e6ae9f8230d1 None admin] PATH_INFO: `/v3/projects/67b6f1fcd7ad44dc9201552ec8996971` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:14.094673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59178b1-f1ac-4732-9a3a-e6ae9f8230d1 None admin] RBAC: Authorizing `identity:get_project(project_id=67b6f1fcd7ad44dc9201552ec8996971)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:14.096179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59178b1-f1ac-4732-9a3a-e6ae9f8230d1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:14.097916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 582/1184] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:21:13 2022] GET /identity/v3/projects/67b6f1fcd7ad44dc9201552ec8996971 => generated 362 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:21:14.104755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da9a71-33c9-4db2-9358-880c21c2d0da None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:14.208611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da9a71-33c9-4db2-9358-880c21c2d0da None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:14.225824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:14.244948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80acabb-5617-4dda-9137-14d8a49d2863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:14.245589 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80acabb-5617-4dda-9137-14d8a49d2863 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:14.245874 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80acabb-5617-4dda-9137-14d8a49d2863 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:14.246155 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80acabb-5617-4dda-9137-14d8a49d2863 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:14.246955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80acabb-5617-4dda-9137-14d8a49d2863 None admin] RBAC: Adding query filter params (role.id=7342106a255b4bce8f150726602a8b95, scope.project.id=67b6f1fcd7ad44dc9201552ec8996971, user.id=a3a63d4496df45f9802de94f7306505b) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:21:14.247267 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80acabb-5617-4dda-9137-14d8a49d2863 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:14.248761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80acabb-5617-4dda-9137-14d8a49d2863 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:14.254385 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 603/1185] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:21:14 2022] GET /identity/v3/role_assignments?user.id=a3a63d4496df45f9802de94f7306505b&scope.project.id=67b6f1fcd7ad44dc9201552ec8996971&role.id=7342106a255b4bce8f150726602a8b95 => generated 260 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:21:16.270567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73776d6d-dc85-465e-9639-0f7a33c8a4c8 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:16.271251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73776d6d-dc85-465e-9639-0f7a33c8a4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:16.271251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73776d6d-dc85-465e-9639-0f7a33c8a4c8 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:16.272599 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 583/1186] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:21:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:21:16.278138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ec122-136c-48b5-8b26-fdca0f647935 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:16.278597 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ec122-136c-48b5-8b26-fdca0f647935 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:16.278719 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ec122-136c-48b5-8b26-fdca0f647935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:16.297872 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-861ec122-136c-48b5-8b26-fdca0f647935 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:21:16.299111 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-861ec122-136c-48b5-8b26-fdca0f647935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:21:16.375872 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 604/1187] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:21:16 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:21:16.399938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73776d6d-dc85-465e-9639-0f7a33c8a4c8 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:16.523046 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73776d6d-dc85-465e-9639-0f7a33c8a4c8 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:16.566376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d24410-3c75-48ba-a3c7-3ced5a6c7b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:16.567046 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d24410-3c75-48ba-a3c7-3ced5a6c7b2c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:16.567331 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d24410-3c75-48ba-a3c7-3ced5a6c7b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:16.567648 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d24410-3c75-48ba-a3c7-3ced5a6c7b2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:16.579797 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d24410-3c75-48ba-a3c7-3ced5a6c7b2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:21:16.584871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-78d24410-3c75-48ba-a3c7-3ced5a6c7b2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:21:16.584871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:21:16.584871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:21:16.584871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:21:16.584871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:21:16.584871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:21:16.584871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:21:16.584871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:21:16.584871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:21:16.584871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:21:16.584871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:21:16.584871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:21:16.584871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:21:16.584871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:21:16.584871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:21:16.584871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:21:16.586089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:21:16.586089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:21:16.586089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:21:16.586089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:21:16.586089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:21:16.586089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:21:16.586089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:21:16.586089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:21:16.586089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:21:16.586089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:21:16.586089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:21:16.586089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:21:16.586089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:21:16.586089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:21:16.586089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:21:16.586089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:21:16.592492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:21:16.592492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:21:16.592492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:21:16.592492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:21:16.592492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:21:16.592492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:21:16.592492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:21:16.592492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:21:16.592492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:21:16.592492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:21:16.592492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:21:16.592492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:21:16.592492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 584/1188] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:21:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:21:16.593359 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ec122-136c-48b5-8b26-fdca0f647935 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:16.709920 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ec122-136c-48b5-8b26-fdca0f647935 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:16.722283 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:16.750873 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb891d6-c8e4-4975-81b7-a812f13dc64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:16.751523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb891d6-c8e4-4975-81b7-a812f13dc64c None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:16.751815 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb891d6-c8e4-4975-81b7-a812f13dc64c None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:16.752132 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb891d6-c8e4-4975-81b7-a812f13dc64c None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:16.752888 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb891d6-c8e4-4975-81b7-a812f13dc64c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:21:16.753201 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb891d6-c8e4-4975-81b7-a812f13dc64c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:16.754674 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb891d6-c8e4-4975-81b7-a812f13dc64c None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:16.764888 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 605/1189] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:21:16 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:21:16.771605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d24410-3c75-48ba-a3c7-3ced5a6c7b2c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:16.838565 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:16.890022 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d24410-3c75-48ba-a3c7-3ced5a6c7b2c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:16.928373 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17420eb4-3580-4d56-b1dc-5edccd63d439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:16.929037 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17420eb4-3580-4d56-b1dc-5edccd63d439 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:16.929322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17420eb4-3580-4d56-b1dc-5edccd63d439 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:16.929595 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17420eb4-3580-4d56-b1dc-5edccd63d439 None admin] PATH_INFO: `/v3/users/a3a63d4496df45f9802de94f7306505b` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:16.930933 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17420eb4-3580-4d56-b1dc-5edccd63d439 None admin] RBAC: Authorizing `identity:get_user(user_id=a3a63d4496df45f9802de94f7306505b)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:16.932482 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17420eb4-3580-4d56-b1dc-5edccd63d439 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:16.934208 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 585/1190] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:21:16 2022] GET /identity/v3/users/a3a63d4496df45f9802de94f7306505b => generated 312 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:21:16.940658 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb891d6-c8e4-4975-81b7-a812f13dc64c None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:17.044806 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb891d6-c8e4-4975-81b7-a812f13dc64c None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:17.080734 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe78e6-f80e-4368-ac36-fe3f08c4a5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:17.081402 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe78e6-f80e-4368-ac36-fe3f08c4a5be None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:17.081707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe78e6-f80e-4368-ac36-fe3f08c4a5be None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:17.081986 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe78e6-f80e-4368-ac36-fe3f08c4a5be None admin] PATH_INFO: `/v3/projects/67b6f1fcd7ad44dc9201552ec8996971` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:17.084869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe78e6-f80e-4368-ac36-fe3f08c4a5be None admin] RBAC: Authorizing `identity:get_project(project_id=67b6f1fcd7ad44dc9201552ec8996971)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:17.086356 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe78e6-f80e-4368-ac36-fe3f08c4a5be None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:17.088140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 606/1191] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:21:16 2022] GET /identity/v3/projects/67b6f1fcd7ad44dc9201552ec8996971 => generated 362 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:21:17.095203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17420eb4-3580-4d56-b1dc-5edccd63d439 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:17.218535 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17420eb4-3580-4d56-b1dc-5edccd63d439 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:17.240890 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:17.256460 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2ea72-94bb-4c4e-b416-add89252ef12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:17.257164 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2ea72-94bb-4c4e-b416-add89252ef12 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:17.257437 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2ea72-94bb-4c4e-b416-add89252ef12 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:17.257712 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2ea72-94bb-4c4e-b416-add89252ef12 None admin] PATH_INFO: `/v3/projects/67b6f1fcd7ad44dc9201552ec8996971/users/a3a63d4496df45f9802de94f7306505b/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:17.259724 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d2ea72-94bb-4c4e-b416-add89252ef12 None admin] RBAC: Authorizing `identity:create_grant(project_id=67b6f1fcd7ad44dc9201552ec8996971, user_id=a3a63d4496df45f9802de94f7306505b, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:17.261296 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d2ea72-94bb-4c4e-b416-add89252ef12 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:17.275928 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 586/1192] 10.222.0.121 () {58 vars in 1444 bytes} [Thu Aug 4 10:21:17 2022] PUT /identity/v3/projects/67b6f1fcd7ad44dc9201552ec8996971/users/a3a63d4496df45f9802de94f7306505b/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:21:19.312479 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44076ce6-3cec-4469-b3c3-2c18ff4c4f9f None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:19.313180 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44076ce6-3cec-4469-b3c3-2c18ff4c4f9f None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:19.313180 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44076ce6-3cec-4469-b3c3-2c18ff4c4f9f None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:19.314457 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 607/1193] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:21:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:21:19.321125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f7470-3920-4dfb-954d-5be8d4edf449 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:19.321779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f7470-3920-4dfb-954d-5be8d4edf449 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:19.321779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f7470-3920-4dfb-954d-5be8d4edf449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:19.344563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-616f7470-3920-4dfb-954d-5be8d4edf449 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:21:19.345826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-616f7470-3920-4dfb-954d-5be8d4edf449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:21:19.426379 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 587/1194] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:21:19 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:21:19.450803 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44076ce6-3cec-4469-b3c3-2c18ff4c4f9f None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:19.496488 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:19.563159 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44076ce6-3cec-4469-b3c3-2c18ff4c4f9f None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:19.601377 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f75a0-0b1b-4ce3-8997-3f0b10417e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:19.602043 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f75a0-0b1b-4ce3-8997-3f0b10417e68 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:19.602330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f75a0-0b1b-4ce3-8997-3f0b10417e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:19.602607 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f75a0-0b1b-4ce3-8997-3f0b10417e68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:19.613906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f75a0-0b1b-4ce3-8997-3f0b10417e68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:21:19.618907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-366f75a0-0b1b-4ce3-8997-3f0b10417e68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:21:19.618907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:21:19.618907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:21:19.618907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:21:19.618907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:21:19.618907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:21:19.618907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:21:19.618907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:21:19.618907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:21:19.618907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:21:19.618907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:21:19.618907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:21:19.618907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:21:19.618907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:21:19.618907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:21:19.618907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:21:19.620044 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:21:19.620044 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:21:19.620044 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:21:19.620044 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:21:19.620044 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:21:19.620044 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:21:19.620044 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:21:19.620044 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:21:19.620044 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:21:19.620044 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:21:19.620044 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:21:19.620044 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:21:19.620044 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:21:19.620044 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:21:19.620044 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:21:19.620044 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:21:19.620958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:21:19.620958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:21:19.620958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:21:19.620958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:21:19.620958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:21:19.620958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:21:19.620958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:21:19.620958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:21:19.620958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:21:19.620958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:21:19.620958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:21:19.620958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:21:19.620958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 608/1195] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:21:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:21:19.626818 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f7470-3920-4dfb-954d-5be8d4edf449 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:19.729762 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f7470-3920-4dfb-954d-5be8d4edf449 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:19.741974 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:19.768336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62bb5cc-37ed-4bb2-95df-f6aa89dd67c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:19.769015 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62bb5cc-37ed-4bb2-95df-f6aa89dd67c0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:19.769295 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62bb5cc-37ed-4bb2-95df-f6aa89dd67c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:19.769568 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62bb5cc-37ed-4bb2-95df-f6aa89dd67c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:19.770308 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62bb5cc-37ed-4bb2-95df-f6aa89dd67c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:21:19.770731 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62bb5cc-37ed-4bb2-95df-f6aa89dd67c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:19.772154 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62bb5cc-37ed-4bb2-95df-f6aa89dd67c0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:19.780815 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 588/1196] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:21:19 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:21:19.787029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f75a0-0b1b-4ce3-8997-3f0b10417e68 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:19.886886 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:19.890184 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f75a0-0b1b-4ce3-8997-3f0b10417e68 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:19.928590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401be78-746e-4380-a1bb-2d4e17130b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:19.929251 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401be78-746e-4380-a1bb-2d4e17130b74 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:19.929533 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401be78-746e-4380-a1bb-2d4e17130b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:19.929805 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401be78-746e-4380-a1bb-2d4e17130b74 None admin] PATH_INFO: `/v3/users/a3a63d4496df45f9802de94f7306505b` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:19.931018 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401be78-746e-4380-a1bb-2d4e17130b74 None admin] RBAC: Authorizing `identity:get_user(user_id=a3a63d4496df45f9802de94f7306505b)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:19.932537 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401be78-746e-4380-a1bb-2d4e17130b74 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:19.934204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 609/1197] 10.222.0.121 () {56 vars in 1182 bytes} [Thu Aug 4 10:21:19 2022] GET /identity/v3/users/a3a63d4496df45f9802de94f7306505b => generated 312 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:21:19.940906 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62bb5cc-37ed-4bb2-95df-f6aa89dd67c0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:20.046261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62bb5cc-37ed-4bb2-95df-f6aa89dd67c0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:20.084357 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65b0d2-0ddc-4262-83df-4a18c0f59656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:20.084995 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65b0d2-0ddc-4262-83df-4a18c0f59656 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:20.085282 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65b0d2-0ddc-4262-83df-4a18c0f59656 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:20.085561 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65b0d2-0ddc-4262-83df-4a18c0f59656 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:20.129382 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:20.161923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a65b0d2-0ddc-4262-83df-4a18c0f59656 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:20.163302 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a65b0d2-0ddc-4262-83df-4a18c0f59656 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:20.270759 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 589/1198] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:21:19 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:21:20.280443 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401be78-746e-4380-a1bb-2d4e17130b74 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:20.391751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401be78-746e-4380-a1bb-2d4e17130b74 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:20.428701 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34055a-ed01-4707-a442-dcc501b49efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:20.429268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34055a-ed01-4707-a442-dcc501b49efb None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:20.429551 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34055a-ed01-4707-a442-dcc501b49efb None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:20.429831 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34055a-ed01-4707-a442-dcc501b49efb None admin] PATH_INFO: `/v3/projects/67b6f1fcd7ad44dc9201552ec8996971` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:20.431093 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b34055a-ed01-4707-a442-dcc501b49efb None admin] RBAC: Authorizing `identity:get_project(project_id=67b6f1fcd7ad44dc9201552ec8996971)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:20.432614 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b34055a-ed01-4707-a442-dcc501b49efb None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:20.434357 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 610/1199] 10.222.0.121 () {56 vars in 1191 bytes} [Thu Aug 4 10:21:20 2022] GET /identity/v3/projects/67b6f1fcd7ad44dc9201552ec8996971 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:21:20.441121 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65b0d2-0ddc-4262-83df-4a18c0f59656 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:21:20.517836 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:20.550094 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65b0d2-0ddc-4262-83df-4a18c0f59656 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:21:20.588345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29fac0-1050-4814-a3b9-a826aab590a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:21:20.588983 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29fac0-1050-4814-a3b9-a826aab590a5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:21:20.589272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29fac0-1050-4814-a3b9-a826aab590a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:21:20.589545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29fac0-1050-4814-a3b9-a826aab590a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:21:20.590355 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29fac0-1050-4814-a3b9-a826aab590a5 None admin] RBAC: Adding query filter params (role.id=7342106a255b4bce8f150726602a8b95, scope.project.id=67b6f1fcd7ad44dc9201552ec8996971, user.id=a3a63d4496df45f9802de94f7306505b) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:21:20.590666 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29fac0-1050-4814-a3b9-a826aab590a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:21:20.592162 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29fac0-1050-4814-a3b9-a826aab590a5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:21:20.598565 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 590/1200] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:21:20 2022] GET /identity/v3/role_assignments?user.id=a3a63d4496df45f9802de94f7306505b&scope.project.id=67b6f1fcd7ad44dc9201552ec8996971&role.id=7342106a255b4bce8f150726602a8b95 => generated 610 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:21:30.424178 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=82 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:21:30.596124 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=32 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:18.750476 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87024a-6ec5-4aa5-97fd-4439b4d1a5a7 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:18.751185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87024a-6ec5-4aa5-97fd-4439b4d1a5a7 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:18.751185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87024a-6ec5-4aa5-97fd-4439b4d1a5a7 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:18.752905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 611/1201] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:22:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:22:18.759597 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761819e-2bcc-466d-adf1-d6accb3ce6fa None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:18.760185 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761819e-2bcc-466d-adf1-d6accb3ce6fa None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:18.760275 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761819e-2bcc-466d-adf1-d6accb3ce6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:18.781814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f761819e-2bcc-466d-adf1-d6accb3ce6fa None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:22:18.783686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f761819e-2bcc-466d-adf1-d6accb3ce6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:22:18.888405 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 591/1202] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:22:18 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:22:18.902845 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87024a-6ec5-4aa5-97fd-4439b4d1a5a7 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:19.046273 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87024a-6ec5-4aa5-97fd-4439b4d1a5a7 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:19.097840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aceddd-7d11-40b2-ae13-6c18ae10536f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:19.098736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aceddd-7d11-40b2-ae13-6c18ae10536f None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:19.099087 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aceddd-7d11-40b2-ae13-6c18ae10536f None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:19.099436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aceddd-7d11-40b2-ae13-6c18ae10536f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:19.113708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aceddd-7d11-40b2-ae13-6c18ae10536f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:19.115572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aceddd-7d11-40b2-ae13-6c18ae10536f None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:19.138760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-19aceddd-7d11-40b2-ae13-6c18ae10536f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:19.138760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:19.138760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:19.138760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:19.138760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:19.138760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:19.138760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:19.138760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:19.138760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:19.138760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:19.138760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:19.138760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:19.138760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:22:19.138760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:22:19.138760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:22:19.140037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:22:19.140037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:19.140037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:19.140037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:19.140037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:19.140037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:19.140037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:19.140037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:19.140037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:19.140037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:19.140037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:19.140037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:19.140037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:19.140037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:19.140037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:19.140037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:19.140037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:19.141094 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:22:19.141094 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:22:19.141094 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:19.141094 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:22:19.141094 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 612/1203] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:22:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:19.146957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761819e-2bcc-466d-adf1-d6accb3ce6fa None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:19.295666 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761819e-2bcc-466d-adf1-d6accb3ce6fa None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:19.344909 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:19.355230 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb597bca-c92b-40d6-958e-577a941d42a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:19.355923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb597bca-c92b-40d6-958e-577a941d42a4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:19.356270 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb597bca-c92b-40d6-958e-577a941d42a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:19.356560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb597bca-c92b-40d6-958e-577a941d42a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:19.357417 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb597bca-c92b-40d6-958e-577a941d42a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:19.357742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb597bca-c92b-40d6-958e-577a941d42a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:19.359367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb597bca-c92b-40d6-958e-577a941d42a4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:19.377513 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 592/1204] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:22:19 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:19.384716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aceddd-7d11-40b2-ae13-6c18ae10536f None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:19.468012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:19.545050 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aceddd-7d11-40b2-ae13-6c18ae10536f None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:19.610724 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e845d7b-0d11-4318-9cb8-30651c491221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:19.611472 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e845d7b-0d11-4318-9cb8-30651c491221 None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:19.612162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e845d7b-0d11-4318-9cb8-30651c491221 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:19.612431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e845d7b-0d11-4318-9cb8-30651c491221 None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:19.613157 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e845d7b-0d11-4318-9cb8-30651c491221 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:19.615062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e845d7b-0d11-4318-9cb8-30651c491221 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:19.652331 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 613/1205] 10.222.0.121 () {60 vars in 1137 bytes} [Thu Aug 4 10:22:19 2022] POST /identity/v3/users => generated 252 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:22:22.052709 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a958820-bd4c-4434-ab32-45cc009d0d6a None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:22.053344 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a958820-bd4c-4434-ab32-45cc009d0d6a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:22.053344 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a958820-bd4c-4434-ab32-45cc009d0d6a None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:22.054705 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 593/1206] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:22:22 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:22:22.061499 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d7a6f-7bfa-4239-a047-61a541a649f4 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:22.062078 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d7a6f-7bfa-4239-a047-61a541a649f4 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:22.062078 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d7a6f-7bfa-4239-a047-61a541a649f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:22.085531 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-0a0d7a6f-7bfa-4239-a047-61a541a649f4 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:22:22.087046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-0a0d7a6f-7bfa-4239-a047-61a541a649f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:22:22.169700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 614/1207] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:22:22 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:22:22.197217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a958820-bd4c-4434-ab32-45cc009d0d6a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:22.342613 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a958820-bd4c-4434-ab32-45cc009d0d6a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:22.384256 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34a000-7bad-41e3-88bf-8d1bab35eba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:22.384931 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34a000-7bad-41e3-88bf-8d1bab35eba1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:22.385215 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34a000-7bad-41e3-88bf-8d1bab35eba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:22.385513 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34a000-7bad-41e3-88bf-8d1bab35eba1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:22.398823 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34a000-7bad-41e3-88bf-8d1bab35eba1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:22:22.405309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-cd34a000-7bad-41e3-88bf-8d1bab35eba1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:22:22.405309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:22.405309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:22.405309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:22.405309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:22.405309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:22.405309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:22.405309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:22.405309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:22.405309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:22.405309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:22.405309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:22.405309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:22:22.405309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:22:22.405309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:22:22.405309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:22:22.411410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:22:22.411410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:22:22.411410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:22:22.411410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:22:22.411410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:22.411410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:22.411410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:22.411410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:22.411410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:22.411410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:22.411410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:22.411410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:22.411410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:22.411410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:22.411410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:22.411410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:22.413311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:22.413311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:22.413311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:22.413311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:22.413311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:22:22.413311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:22:22.413311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:22:22.413311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:22:22.413311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:22:22.413311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:22:22.413311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:22:22.413311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:22:22.413311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 594/1208] 10.222.0.121 () {56 vars in 1107 bytes} [Thu Aug 4 10:22:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:22.415670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d7a6f-7bfa-4239-a047-61a541a649f4 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:22.471028 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:22.569815 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d7a6f-7bfa-4239-a047-61a541a649f4 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:22.619351 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05941073-1bc6-42f4-a188-feeabef76b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:22.620461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05941073-1bc6-42f4-a188-feeabef76b4d None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:22.620915 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05941073-1bc6-42f4-a188-feeabef76b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:22.621270 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05941073-1bc6-42f4-a188-feeabef76b4d None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:22.622009 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05941073-1bc6-42f4-a188-feeabef76b4d None admin] RBAC: Adding query filter params (name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:22.622332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05941073-1bc6-42f4-a188-feeabef76b4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:22.623876 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05941073-1bc6-42f4-a188-feeabef76b4d None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:22.635713 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 615/1209] 10.222.0.121 () {56 vars in 1108 bytes} [Thu Aug 4 10:22:22 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:22.642636 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34a000-7bad-41e3-88bf-8d1bab35eba1 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:22.724928 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:22.797550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34a000-7bad-41e3-88bf-8d1bab35eba1 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:22.847309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963bad0-72b3-4e55-9972-1236989ba26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:22.848070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963bad0-72b3-4e55-9972-1236989ba26f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:22.848377 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963bad0-72b3-4e55-9972-1236989ba26f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:22.848663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963bad0-72b3-4e55-9972-1236989ba26f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:22.868974 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963bad0-72b3-4e55-9972-1236989ba26f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:22.870582 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963bad0-72b3-4e55-9972-1236989ba26f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:22.887808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-2963bad0-72b3-4e55-9972-1236989ba26f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:22.887808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:22.887808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:22.887808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:22.887808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:22.887808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:22.887808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:22.887808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:22.887808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:22.887808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:22.887808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:22.887808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:22.887808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:22:22.887808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:22:22.887808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:22:22.888996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:22:22.888996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:22.888996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:22.888996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:22.888996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:22.888996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:22.888996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:22.888996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:22.888996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:22.888996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:22.888996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:22.888996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:22.888996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:22.888996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:22.888996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:22.888996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:22.888996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:22.890011 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:22:22.890011 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:22:22.890011 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:22.890011 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:22:22.890011 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 595/1210] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:22:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:22.896346 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05941073-1bc6-42f4-a188-feeabef76b4d None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:23.031878 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:23.035600 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05941073-1bc6-42f4-a188-feeabef76b4d None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:23.078440 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1746bdc-4e15-43d3-9210-1689d136903a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:23.079390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1746bdc-4e15-43d3-9210-1689d136903a None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:23.079504 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1746bdc-4e15-43d3-9210-1689d136903a None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:23.079760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1746bdc-4e15-43d3-9210-1689d136903a None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:23.080556 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1746bdc-4e15-43d3-9210-1689d136903a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:23.080852 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1746bdc-4e15-43d3-9210-1689d136903a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:23.082371 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1746bdc-4e15-43d3-9210-1689d136903a None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:23.099457 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 616/1211] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:22:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:23.105238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963bad0-72b3-4e55-9972-1236989ba26f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:23.240607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963bad0-72b3-4e55-9972-1236989ba26f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:23.242339 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:23.285437 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007e7a0-aa06-45a6-b3f0-8efb58f79bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:23.286186 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007e7a0-aa06-45a6-b3f0-8efb58f79bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:23.286504 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007e7a0-aa06-45a6-b3f0-8efb58f79bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:23.286752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007e7a0-aa06-45a6-b3f0-8efb58f79bc8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:23.296171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b007e7a0-aa06-45a6-b3f0-8efb58f79bc8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:23.297918 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b007e7a0-aa06-45a6-b3f0-8efb58f79bc8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:23.311307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-b007e7a0-aa06-45a6-b3f0-8efb58f79bc8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 10:22:23.311307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:23.311307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:23.311307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:23.311307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:23.311307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:23.311307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:23.311307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:23.311307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:23.311307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:23.311307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:23.311307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:23.311307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:22:23.311307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:22:23.311307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:22:23.311307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:22:23.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:23.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:23.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:22:23.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:22:23.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:22:23.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:22:23.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:23.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:23.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:23.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:23.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:23.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:23.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:23.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:23.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:23.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:23.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:23.313642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:23.315229 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:23.315229 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:23.315229 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:22:23.315229 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:22:23.315229 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:22:23.315229 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:22:23.315229 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:22:23.315229 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:22:23.315229 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 10:22:23.315229 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:22:23.316183 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 596/1212] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:22:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:23.321899 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1746bdc-4e15-43d3-9210-1689d136903a None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:23.435908 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1746bdc-4e15-43d3-9210-1689d136903a None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:23.475253 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcad56-c6ca-48fe-a128-9bcf75396084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:23.476045 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcad56-c6ca-48fe-a128-9bcf75396084 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:23.476387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcad56-c6ca-48fe-a128-9bcf75396084 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:23.476853 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcad56-c6ca-48fe-a128-9bcf75396084 None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:23.477489 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dcad56-c6ca-48fe-a128-9bcf75396084 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:23.477804 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dcad56-c6ca-48fe-a128-9bcf75396084 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:23.479406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dcad56-c6ca-48fe-a128-9bcf75396084 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:23.500026 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 617/1213] 10.222.0.121 () {56 vars in 1148 bytes} [Thu Aug 4 10:22:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:23.506170 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007e7a0-aa06-45a6-b3f0-8efb58f79bc8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:23.625172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007e7a0-aa06-45a6-b3f0-8efb58f79bc8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:23.670824 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1becad-0a4a-46b7-a818-591567ac6c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:23.672091 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1becad-0a4a-46b7-a818-591567ac6c56 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:23.672684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1becad-0a4a-46b7-a818-591567ac6c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:23.673266 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1becad-0a4a-46b7-a818-591567ac6c56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:23.715841 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:23.763912 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1becad-0a4a-46b7-a818-591567ac6c56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:23.765462 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1becad-0a4a-46b7-a818-591567ac6c56 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:23.877407 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 597/1214] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:22:23 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:22:23.883556 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcad56-c6ca-48fe-a128-9bcf75396084 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:23.905716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:24.012144 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcad56-c6ca-48fe-a128-9bcf75396084 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:24.063162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e1905-8209-437c-950f-2f805d58263e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:24.063760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e1905-8209-437c-950f-2f805d58263e None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:24.064089 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e1905-8209-437c-950f-2f805d58263e None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:24.064372 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e1905-8209-437c-950f-2f805d58263e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:24.078561 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e1905-8209-437c-950f-2f805d58263e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:24.080318 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e1905-8209-437c-950f-2f805d58263e None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:24.103108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-151e1905-8209-437c-950f-2f805d58263e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:24.103108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:24.103108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:24.103108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:24.103108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:24.103108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:24.103108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:24.103108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:24.103108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:24.103108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:24.103108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:24.103108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:24.103108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:22:24.103108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:22:24.103108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:22:24.104615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:22:24.104615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:24.104615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:24.104615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:24.104615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:24.104615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:24.104615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:24.104615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:24.104615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:24.104615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:24.104615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:24.104615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:24.104615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:24.104615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:24.104615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:24.104615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:24.104615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:24.105775 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:22:24.105775 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:22:24.105775 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:24.105775 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:22:24.105775 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 618/1215] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:22:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:24.112156 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1becad-0a4a-46b7-a818-591567ac6c56 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:24.194407 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:24.240054 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1becad-0a4a-46b7-a818-591567ac6c56 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:24.296784 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1a51f-edcf-4d02-9afe-26c764e513fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:24.297765 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1a51f-edcf-4d02-9afe-26c764e513fd None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:24.298116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1a51f-edcf-4d02-9afe-26c764e513fd None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:24.298497 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1a51f-edcf-4d02-9afe-26c764e513fd None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:24.299628 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1a51f-edcf-4d02-9afe-26c764e513fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:24.300127 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1a51f-edcf-4d02-9afe-26c764e513fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:24.302362 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1a51f-edcf-4d02-9afe-26c764e513fd None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:24.325183 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 598/1216] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:22:24 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:24.334386 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e1905-8209-437c-950f-2f805d58263e None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:24.420797 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:24.458531 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e1905-8209-437c-950f-2f805d58263e None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:24.508390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298536a-f3d8-40cf-a6f2-2ad989e2e581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:24.509188 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298536a-f3d8-40cf-a6f2-2ad989e2e581 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:24.509555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298536a-f3d8-40cf-a6f2-2ad989e2e581 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:24.509830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298536a-f3d8-40cf-a6f2-2ad989e2e581 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:24.518728 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6298536a-f3d8-40cf-a6f2-2ad989e2e581 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:24.520930 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6298536a-f3d8-40cf-a6f2-2ad989e2e581 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:24.535602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-6298536a-f3d8-40cf-a6f2-2ad989e2e581 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:22:24.535602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:24.535602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:24.535602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:24.535602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:24.535602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:24.535602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:24.535602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:24.535602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:24.535602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:24.535602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:24.535602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:24.535602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:22:24.535602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:22:24.535602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:22:24.537012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:22:24.537012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:24.537012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:24.537012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:24.537012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:24.537012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:24.537012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:24.537012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:24.537012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:24.537012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:24.537012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:24.537012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:24.537012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:24.537012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:24.537012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:24.537012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:24.537012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:24.537982 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:22:24.537982 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:22:24.537982 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:22:24.537982 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:22:24.537982 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:22:24.537982 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:22:24.537982 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:22:24.537982 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:22:24.537982 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 619/1217] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:22:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:22:24.543376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1a51f-edcf-4d02-9afe-26c764e513fd None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:24.681842 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1a51f-edcf-4d02-9afe-26c764e513fd None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:24.709998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:24.723902 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ffd58-41bd-4ecf-a5bd-cec2e4b3f0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:24.724605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ffd58-41bd-4ecf-a5bd-cec2e4b3f0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:24.724856 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ffd58-41bd-4ecf-a5bd-cec2e4b3f0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:24.725163 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ffd58-41bd-4ecf-a5bd-cec2e4b3f0d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:24.725974 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ffd58-41bd-4ecf-a5bd-cec2e4b3f0d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:24.726279 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ffd58-41bd-4ecf-a5bd-cec2e4b3f0d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:24.727868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ffd58-41bd-4ecf-a5bd-cec2e4b3f0d7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:24.743723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 599/1218] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:22:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:24.750804 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298536a-f3d8-40cf-a6f2-2ad989e2e581 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:24.871712 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298536a-f3d8-40cf-a6f2-2ad989e2e581 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:24.907488 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:24.915699 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b495346-6ac6-4014-986e-c69a35b737aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:24.916418 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b495346-6ac6-4014-986e-c69a35b737aa None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:24.916734 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b495346-6ac6-4014-986e-c69a35b737aa None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:24.917049 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b495346-6ac6-4014-986e-c69a35b737aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:24.918089 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b495346-6ac6-4014-986e-c69a35b737aa None admin] RBAC: Adding query filter params (role.id=2edb704d42834db6acd295fa3a8be5d5, scope.project.id=22768aba760b4307aab131d1312b2bf6, user.id=01c7cfbe911c4378857848966f4f8d98) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:24.918469 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b495346-6ac6-4014-986e-c69a35b737aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:24.920307 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b495346-6ac6-4014-986e-c69a35b737aa None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:24.927664 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 620/1219] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:22:24 2022] GET /identity/v3/role_assignments?user.id=01c7cfbe911c4378857848966f4f8d98&scope.project.id=22768aba760b4307aab131d1312b2bf6&role.id=2edb704d42834db6acd295fa3a8be5d5 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:27.157082 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcfcc7-bcdc-4903-89e0-0af80d45b7c1 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:27.157818 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcfcc7-bcdc-4903-89e0-0af80d45b7c1 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:27.157818 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcfcc7-bcdc-4903-89e0-0af80d45b7c1 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:27.159071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 600/1220] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:22:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:22:27.165700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f29d1b-e5f8-48c3-a023-b0d1f7f088bd None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:27.166307 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f29d1b-e5f8-48c3-a023-b0d1f7f088bd None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:27.166307 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f29d1b-e5f8-48c3-a023-b0d1f7f088bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:27.186605 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-48f29d1b-e5f8-48c3-a023-b0d1f7f088bd None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:22:27.188043 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-48f29d1b-e5f8-48c3-a023-b0d1f7f088bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:22:27.264979 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 621/1221] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:22:27 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:22:27.290863 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdcfcc7-bcdc-4903-89e0-0af80d45b7c1 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:27.414446 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdcfcc7-bcdc-4903-89e0-0af80d45b7c1 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:27.456533 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43bf6a-dc81-472e-8c67-ac5c70b894b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:27.457258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43bf6a-dc81-472e-8c67-ac5c70b894b4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:27.457642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43bf6a-dc81-472e-8c67-ac5c70b894b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:27.457991 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43bf6a-dc81-472e-8c67-ac5c70b894b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:27.470700 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43bf6a-dc81-472e-8c67-ac5c70b894b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:22:27.476500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-8b43bf6a-dc81-472e-8c67-ac5c70b894b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:22:27.476500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:27.476500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:27.476500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:27.476500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:27.476500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:27.476500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:27.476500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:27.476500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:27.476500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:27.476500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:27.476500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:27.476500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:22:27.476500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:22:27.476500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:22:27.476500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:22:27.477830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:22:27.477830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:22:27.477830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:22:27.477830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:22:27.477830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:27.477830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:27.477830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:27.477830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:27.477830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:27.477830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:27.477830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:27.477830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:27.477830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:27.477830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:27.477830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:27.477830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:27.478924 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:27.478924 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:27.478924 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:27.478924 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:27.478924 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:22:27.478924 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:22:27.478924 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:22:27.478924 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:22:27.478924 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:22:27.478924 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:22:27.478924 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:22:27.478924 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:22:27.483563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 601/1222] 10.222.0.121 () {56 vars in 1107 bytes} [Thu Aug 4 10:22:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:27.484759 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f29d1b-e5f8-48c3-a023-b0d1f7f088bd None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:27.604168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f29d1b-e5f8-48c3-a023-b0d1f7f088bd None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:27.634800 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:27.648153 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d5318-3be4-4cd4-ae28-52e2842e1523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:27.648977 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d5318-3be4-4cd4-ae28-52e2842e1523 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:27.649339 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d5318-3be4-4cd4-ae28-52e2842e1523 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:27.649677 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d5318-3be4-4cd4-ae28-52e2842e1523 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:27.650486 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d5318-3be4-4cd4-ae28-52e2842e1523 None admin] RBAC: Adding query filter params (name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:27.650870 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d5318-3be4-4cd4-ae28-52e2842e1523 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:27.652541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d5318-3be4-4cd4-ae28-52e2842e1523 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:27.662562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 622/1223] 10.222.0.121 () {56 vars in 1108 bytes} [Thu Aug 4 10:22:27 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:27.669243 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43bf6a-dc81-472e-8c67-ac5c70b894b4 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:27.720521 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:27.786016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43bf6a-dc81-472e-8c67-ac5c70b894b4 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:27.826171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379d035-37d1-45c6-92db-205d1ea405d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:27.826915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379d035-37d1-45c6-92db-205d1ea405d1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:27.827244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379d035-37d1-45c6-92db-205d1ea405d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:27.827546 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379d035-37d1-45c6-92db-205d1ea405d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:27.839218 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379d035-37d1-45c6-92db-205d1ea405d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:27.840960 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379d035-37d1-45c6-92db-205d1ea405d1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:27.858509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-6379d035-37d1-45c6-92db-205d1ea405d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:27.858509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:27.858509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:27.858509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:27.858509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:27.858509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:27.858509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:27.858509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:27.858509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:27.858509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:27.858509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:27.858509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:27.858509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:22:27.858509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:22:27.858509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:22:27.859741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:22:27.859741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:27.859741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:27.859741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:27.859741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:27.859741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:27.859741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:27.859741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:27.859741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:27.859741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:27.859741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:27.859741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:27.859741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:27.859741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:27.859741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:27.859741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:27.859741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:27.861518 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:22:27.861518 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:22:27.861518 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:27.861518 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:22:27.861518 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 602/1224] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:22:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:27.867395 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d5318-3be4-4cd4-ae28-52e2842e1523 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:27.992601 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d5318-3be4-4cd4-ae28-52e2842e1523 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:28.031152 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452be6f0-1414-4cc8-9750-60c4a93b17f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:28.031820 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452be6f0-1414-4cc8-9750-60c4a93b17f4 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:28.032137 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452be6f0-1414-4cc8-9750-60c4a93b17f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:28.032452 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452be6f0-1414-4cc8-9750-60c4a93b17f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:28.033215 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452be6f0-1414-4cc8-9750-60c4a93b17f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:28.033585 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452be6f0-1414-4cc8-9750-60c4a93b17f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:28.035117 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452be6f0-1414-4cc8-9750-60c4a93b17f4 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:28.053322 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 623/1225] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:22:27 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:22:28.054753 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379d035-37d1-45c6-92db-205d1ea405d1 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:28.172350 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:28.180038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379d035-37d1-45c6-92db-205d1ea405d1 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:28.219824 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7479c99-1907-4dba-9a7b-b3d099848c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:28.220391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7479c99-1907-4dba-9a7b-b3d099848c5f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:28.220653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7479c99-1907-4dba-9a7b-b3d099848c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:28.220934 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7479c99-1907-4dba-9a7b-b3d099848c5f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:28.229676 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7479c99-1907-4dba-9a7b-b3d099848c5f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:28.231179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7479c99-1907-4dba-9a7b-b3d099848c5f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:28.244447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-a7479c99-1907-4dba-9a7b-b3d099848c5f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 10:22:28.244447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:28.244447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:28.244447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:28.244447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:28.244447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:28.244447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:28.244447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:28.244447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:28.244447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:28.244447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:28.244447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:28.244447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:22:28.244447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:22:28.244447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:22:28.244447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:22:28.246674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:28.246674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:28.246674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:22:28.246674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:22:28.246674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:22:28.246674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:22:28.246674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:28.246674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:28.246674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:28.246674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:28.246674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:28.246674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:28.246674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:28.246674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:28.246674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:28.246674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:28.246674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:28.246674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:28.248002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:28.248002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:28.248002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:22:28.248002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:22:28.248002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:22:28.248002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:22:28.248002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:22:28.248002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:22:28.248002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 10:22:28.248002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:22:28.248002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 603/1226] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:22:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:28.252905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452be6f0-1414-4cc8-9750-60c4a93b17f4 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:28.329255 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:28.399391 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452be6f0-1414-4cc8-9750-60c4a93b17f4 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:28.445901 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3bffc-576a-4e07-b3c4-9bd0fc138426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:28.446572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3bffc-576a-4e07-b3c4-9bd0fc138426 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:28.446860 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3bffc-576a-4e07-b3c4-9bd0fc138426 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:28.447133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3bffc-576a-4e07-b3c4-9bd0fc138426 None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:28.448022 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3bffc-576a-4e07-b3c4-9bd0fc138426 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:28.448358 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3bffc-576a-4e07-b3c4-9bd0fc138426 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:28.449828 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3bffc-576a-4e07-b3c4-9bd0fc138426 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:28.468590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 624/1227] 10.222.0.121 () {56 vars in 1148 bytes} [Thu Aug 4 10:22:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:28.478687 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7479c99-1907-4dba-9a7b-b3d099848c5f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:28.608916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7479c99-1907-4dba-9a7b-b3d099848c5f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:28.648877 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f501c-9662-4763-ad1c-972d6d652af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:28.649597 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f501c-9662-4763-ad1c-972d6d652af7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:28.649881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f501c-9662-4763-ad1c-972d6d652af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:28.650155 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f501c-9662-4763-ad1c-972d6d652af7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:28.663913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f501c-9662-4763-ad1c-972d6d652af7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:28.665739 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f501c-9662-4763-ad1c-972d6d652af7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:28.683781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-db9f501c-9662-4763-ad1c-972d6d652af7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:28.683781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:28.683781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:28.683781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:28.683781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:28.683781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:28.683781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:28.683781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:28.683781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:28.683781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:28.683781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:28.683781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:28.683781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:22:28.683781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:22:28.683781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:22:28.686061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:22:28.686061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:28.686061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:28.686061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:28.686061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:28.686061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:28.686061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:28.686061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:28.686061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:28.686061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:28.686061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:28.686061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:28.686061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:28.686061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:28.686061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:28.686061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:28.686061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:28.687981 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:22:28.687981 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:22:28.687981 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:28.687981 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:22:28.687981 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 604/1228] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:22:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:28.691544 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3bffc-576a-4e07-b3c4-9bd0fc138426 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:28.826941 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3bffc-576a-4e07-b3c4-9bd0fc138426 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:28.830206 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:28.871779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ffe2f-a641-4f7a-8c18-1c6dedea674e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:28.872341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ffe2f-a641-4f7a-8c18-1c6dedea674e None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:28.872622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ffe2f-a641-4f7a-8c18-1c6dedea674e None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:28.872894 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ffe2f-a641-4f7a-8c18-1c6dedea674e None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:28.873830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ffe2f-a641-4f7a-8c18-1c6dedea674e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:28.874136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ffe2f-a641-4f7a-8c18-1c6dedea674e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:28.875596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ffe2f-a641-4f7a-8c18-1c6dedea674e None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:28.893034 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 625/1229] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:22:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:28.899177 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f501c-9662-4763-ad1c-972d6d652af7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:28.910150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:29.012251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f501c-9662-4763-ad1c-972d6d652af7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:29.053907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676a549-7521-4f1b-b872-7ad3acc7b6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:29.054680 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676a549-7521-4f1b-b872-7ad3acc7b6b7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:29.055011 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676a549-7521-4f1b-b872-7ad3acc7b6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:29.055292 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676a549-7521-4f1b-b872-7ad3acc7b6b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:29.063008 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676a549-7521-4f1b-b872-7ad3acc7b6b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:29.064562 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676a549-7521-4f1b-b872-7ad3acc7b6b7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:29.078028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-3676a549-7521-4f1b-b872-7ad3acc7b6b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:22:29.078028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:29.078028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:29.078028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:29.078028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:29.078028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:29.078028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:29.078028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:29.078028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:29.078028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:29.078028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:29.078028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:29.078028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:22:29.078028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:22:29.078028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:22:29.079548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:22:29.079548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:29.079548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:29.079548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:29.079548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:29.079548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:29.079548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:29.079548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:29.079548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:29.079548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:29.079548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:29.079548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:29.079548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:29.079548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:29.079548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:29.079548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:29.079548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:29.080825 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:22:29.080825 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:22:29.080825 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:22:29.080825 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:22:29.080825 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:22:29.080825 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:22:29.080825 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:22:29.080825 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:22:29.080825 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 605/1230] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:22:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:29.086603 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ffe2f-a641-4f7a-8c18-1c6dedea674e None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:29.209460 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ffe2f-a641-4f7a-8c18-1c6dedea674e None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:29.247173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6cac7-a8b1-43b7-add1-9d4aee1a976f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:29.247856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6cac7-a8b1-43b7-add1-9d4aee1a976f None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:29.248191 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6cac7-a8b1-43b7-add1-9d4aee1a976f None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:29.248468 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6cac7-a8b1-43b7-add1-9d4aee1a976f None admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:29.249248 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6cac7-a8b1-43b7-add1-9d4aee1a976f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:29.249560 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6cac7-a8b1-43b7-add1-9d4aee1a976f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:29.251071 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6cac7-a8b1-43b7-add1-9d4aee1a976f None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:29.264540 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 626/1231] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:22:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:29.270984 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676a549-7521-4f1b-b872-7ad3acc7b6b7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:29.349803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:29.392638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676a549-7521-4f1b-b872-7ad3acc7b6b7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:29.433389 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627b51d-72cf-444c-b88f-70d0e5969ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:29.434085 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627b51d-72cf-444c-b88f-70d0e5969ef1 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:29.434367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627b51d-72cf-444c-b88f-70d0e5969ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:29.434642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627b51d-72cf-444c-b88f-70d0e5969ef1 None admin] PATH_INFO: `/v3/projects/22768aba760b4307aab131d1312b2bf6/users/01c7cfbe911c4378857848966f4f8d98/roles/2edb704d42834db6acd295fa3a8be5d5` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:29.454044 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2627b51d-72cf-444c-b88f-70d0e5969ef1 None admin] RBAC: Authorizing `identity:create_grant(project_id=22768aba760b4307aab131d1312b2bf6, user_id=01c7cfbe911c4378857848966f4f8d98, role_id=2edb704d42834db6acd295fa3a8be5d5)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:29.455672 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2627b51d-72cf-444c-b88f-70d0e5969ef1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:29.469066 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 606/1232] 10.222.0.121 () {58 vars in 1444 bytes} [Thu Aug 4 10:22:29 2022] PUT /identity/v3/projects/22768aba760b4307aab131d1312b2bf6/users/01c7cfbe911c4378857848966f4f8d98/roles/2edb704d42834db6acd295fa3a8be5d5 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:22:31.751883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062e0a8-ec61-441a-8da7-9f30830d8999 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:31.752603 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062e0a8-ec61-441a-8da7-9f30830d8999 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:31.752603 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062e0a8-ec61-441a-8da7-9f30830d8999 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:31.754396 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 627/1233] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:22:31 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:22:31.761157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4919e-cd98-41ab-b8e3-f18994187252 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:31.761805 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4919e-cd98-41ab-b8e3-f18994187252 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:31.761805 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4919e-cd98-41ab-b8e3-f18994187252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:31.785034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7df4919e-cd98-41ab-b8e3-f18994187252 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:22:31.786371 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7df4919e-cd98-41ab-b8e3-f18994187252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:22:31.872019 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 607/1234] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:22:31 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:22:31.892775 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062e0a8-ec61-441a-8da7-9f30830d8999 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:31.927344 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:32.022961 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062e0a8-ec61-441a-8da7-9f30830d8999 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:32.064073 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d1c77-6294-4068-834f-87b2f49b1583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:32.064716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d1c77-6294-4068-834f-87b2f49b1583 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:32.065037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d1c77-6294-4068-834f-87b2f49b1583 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:32.065294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d1c77-6294-4068-834f-87b2f49b1583 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:32.077922 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d1c77-6294-4068-834f-87b2f49b1583 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:22:32.083475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-b36d1c77-6294-4068-834f-87b2f49b1583 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:22:32.083475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:32.083475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:32.083475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:32.083475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:32.083475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:32.083475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:32.083475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:32.083475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:32.083475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:32.083475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:32.083475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:32.083475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:22:32.083475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:22:32.083475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:22:32.083475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:22:32.084652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:22:32.084652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:22:32.084652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:22:32.084652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:22:32.084652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:32.084652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:32.084652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:32.084652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:32.084652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:32.084652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:32.084652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:32.084652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:32.084652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:32.084652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:32.084652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:32.084652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:32.085580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:32.085580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:32.085580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:32.085580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:32.085580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:22:32.085580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:22:32.085580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:22:32.085580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:22:32.085580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:22:32.085580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:22:32.085580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 10:22:32.085580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:22:32.085580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 628/1235] 10.222.0.121 () {56 vars in 1107 bytes} [Thu Aug 4 10:22:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:32.092660 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4919e-cd98-41ab-b8e3-f18994187252 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:32.152650 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:32.215789 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4919e-cd98-41ab-b8e3-f18994187252 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:32.257922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a16d2a-d661-42be-b88d-c50c66ba5044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:32.258586 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a16d2a-d661-42be-b88d-c50c66ba5044 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:32.258889 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a16d2a-d661-42be-b88d-c50c66ba5044 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:32.259191 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a16d2a-d661-42be-b88d-c50c66ba5044 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:32.260149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a16d2a-d661-42be-b88d-c50c66ba5044 None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:32.260457 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a16d2a-d661-42be-b88d-c50c66ba5044 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:32.262058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a16d2a-d661-42be-b88d-c50c66ba5044 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:32.272964 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 608/1236] 10.222.0.121 () {56 vars in 1108 bytes} [Thu Aug 4 10:22:32 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:32.279786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d1c77-6294-4068-834f-87b2f49b1583 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:32.377318 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:32.396291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d1c77-6294-4068-834f-87b2f49b1583 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:32.432150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad39f7-8ca2-49d1-9ae8-cca655d06a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:32.432712 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad39f7-8ca2-49d1-9ae8-cca655d06a52 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:32.432998 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad39f7-8ca2-49d1-9ae8-cca655d06a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:32.433273 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad39f7-8ca2-49d1-9ae8-cca655d06a52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:32.445676 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad39f7-8ca2-49d1-9ae8-cca655d06a52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:32.447393 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad39f7-8ca2-49d1-9ae8-cca655d06a52 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:32.462949 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-22ad39f7-8ca2-49d1-9ae8-cca655d06a52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:32.462949 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:32.462949 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:32.462949 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:32.462949 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:32.462949 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:32.462949 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:32.462949 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:32.462949 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:32.462949 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:32.462949 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:32.462949 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:32.462949 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:22:32.462949 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:22:32.462949 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:22:32.464189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:22:32.464189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:32.464189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:32.464189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:32.464189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:32.464189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:32.464189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:32.464189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:32.464189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:32.464189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:32.464189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:32.464189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:32.464189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:32.464189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:32.464189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:32.464189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:32.464189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:32.465274 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:22:32.465274 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:22:32.465274 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:32.465274 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:22:32.465274 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 629/1237] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:22:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:22:32.470930 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a16d2a-d661-42be-b88d-c50c66ba5044 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:32.586254 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a16d2a-d661-42be-b88d-c50c66ba5044 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:32.596810 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:32.626342 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3744317-1ab5-4a98-af7f-09448245597f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:32.627062 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3744317-1ab5-4a98-af7f-09448245597f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:32.627365 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3744317-1ab5-4a98-af7f-09448245597f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:32.627678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3744317-1ab5-4a98-af7f-09448245597f None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:32.628483 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3744317-1ab5-4a98-af7f-09448245597f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:32.628801 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3744317-1ab5-4a98-af7f-09448245597f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:32.630248 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3744317-1ab5-4a98-af7f-09448245597f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:32.644314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 609/1238] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:22:32 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:32.650644 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad39f7-8ca2-49d1-9ae8-cca655d06a52 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:32.765957 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad39f7-8ca2-49d1-9ae8-cca655d06a52 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:32.805344 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cbfdf-8a3a-46de-954b-12ec16bf0f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:32.806029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cbfdf-8a3a-46de-954b-12ec16bf0f39 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:32.806316 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cbfdf-8a3a-46de-954b-12ec16bf0f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:32.806595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cbfdf-8a3a-46de-954b-12ec16bf0f39 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:32.809889 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:32.815812 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1cbfdf-8a3a-46de-954b-12ec16bf0f39 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:32.817598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1cbfdf-8a3a-46de-954b-12ec16bf0f39 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:32.830824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-6a1cbfdf-8a3a-46de-954b-12ec16bf0f39 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 10:22:32.830824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:32.830824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:32.830824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:32.830824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:32.830824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:32.830824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:32.830824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:32.830824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:32.830824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:32.830824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:32.830824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:32.830824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:22:32.830824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:22:32.830824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:22:32.830824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:22:32.832185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:32.832185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:32.832185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:22:32.832185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:22:32.832185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:22:32.832185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:22:32.832185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:32.832185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:32.832185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:32.832185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:32.832185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:32.832185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:32.832185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:32.832185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:32.832185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:32.832185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:32.832185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:32.832185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:32.833418 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:32.833418 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:32.833418 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:22:32.833418 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:22:32.833418 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:22:32.833418 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:22:32.833418 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:22:32.833418 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:22:32.833418 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 10:22:32.833418 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:22:32.833418 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 630/1239] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:22:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:32.838574 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3744317-1ab5-4a98-af7f-09448245597f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:32.964312 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3744317-1ab5-4a98-af7f-09448245597f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:33.003681 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8b31b-2c9b-43aa-885b-717ce2db7020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:33.004528 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8b31b-2c9b-43aa-885b-717ce2db7020 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:33.004857 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8b31b-2c9b-43aa-885b-717ce2db7020 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:33.005162 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8b31b-2c9b-43aa-885b-717ce2db7020 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:33.006072 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8b31b-2c9b-43aa-885b-717ce2db7020 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:33.006627 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8b31b-2c9b-43aa-885b-717ce2db7020 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:33.008402 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8b31b-2c9b-43aa-885b-717ce2db7020 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:33.030131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 610/1240] 10.222.0.121 () {56 vars in 1148 bytes} [Thu Aug 4 10:22:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:33.037135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cbfdf-8a3a-46de-954b-12ec16bf0f39 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:33.162201 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cbfdf-8a3a-46de-954b-12ec16bf0f39 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:33.219199 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea909e-229f-4643-aba8-82ee9a59d75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:33.220133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea909e-229f-4643-aba8-82ee9a59d75f None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:33.220488 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea909e-229f-4643-aba8-82ee9a59d75f None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:33.220828 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea909e-229f-4643-aba8-82ee9a59d75f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:33.314724 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:33.326219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea909e-229f-4643-aba8-82ee9a59d75f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:33.328881 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea909e-229f-4643-aba8-82ee9a59d75f None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:33.446197 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 631/1241] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:22:33 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:22:33.451459 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8b31b-2c9b-43aa-885b-717ce2db7020 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:33.481239 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:33.564284 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8b31b-2c9b-43aa-885b-717ce2db7020 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:33.607311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e46cf-e8d6-4fef-9aba-75b14b879e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:33.608195 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e46cf-e8d6-4fef-9aba-75b14b879e56 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:33.608495 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e46cf-e8d6-4fef-9aba-75b14b879e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:33.608821 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e46cf-e8d6-4fef-9aba-75b14b879e56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:33.623705 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193e46cf-e8d6-4fef-9aba-75b14b879e56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:33.625319 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193e46cf-e8d6-4fef-9aba-75b14b879e56 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:33.642548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-193e46cf-e8d6-4fef-9aba-75b14b879e56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:33.642548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:33.642548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:33.642548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:33.642548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:33.642548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:33.642548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:33.642548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:33.642548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:33.642548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:33.642548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:33.642548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:33.642548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:22:33.642548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:22:33.642548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:22:33.643804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:22:33.643804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:33.643804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:33.643804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:33.643804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:33.643804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:33.643804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:33.643804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:33.643804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:33.643804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:33.643804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:33.643804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:33.643804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:33.643804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:33.643804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:33.643804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:33.643804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:33.645006 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:22:33.645006 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:22:33.645006 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:33.645006 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:22:33.645006 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 611/1242] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:22:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:33.652308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea909e-229f-4643-aba8-82ee9a59d75f None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:33.769861 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:33.776811 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea909e-229f-4643-aba8-82ee9a59d75f None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:33.817670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ac4e6-81db-46e0-b7ee-31ccc0ad1564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:33.818320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ac4e6-81db-46e0-b7ee-31ccc0ad1564 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:33.818610 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ac4e6-81db-46e0-b7ee-31ccc0ad1564 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:33.818890 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ac4e6-81db-46e0-b7ee-31ccc0ad1564 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:33.819611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ac4e6-81db-46e0-b7ee-31ccc0ad1564 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:33.819920 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ac4e6-81db-46e0-b7ee-31ccc0ad1564 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:33.821369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ac4e6-81db-46e0-b7ee-31ccc0ad1564 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:33.834821 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 632/1243] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:22:33 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:33.840770 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e46cf-e8d6-4fef-9aba-75b14b879e56 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:33.939858 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:33.965329 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e46cf-e8d6-4fef-9aba-75b14b879e56 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:34.006362 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc732d-08a8-4f61-9f32-1b8c0a865126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:34.007062 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc732d-08a8-4f61-9f32-1b8c0a865126 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:34.007350 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc732d-08a8-4f61-9f32-1b8c0a865126 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:34.007633 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc732d-08a8-4f61-9f32-1b8c0a865126 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:34.016917 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc732d-08a8-4f61-9f32-1b8c0a865126 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:34.018566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc732d-08a8-4f61-9f32-1b8c0a865126 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:34.032206 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-a1bc732d-08a8-4f61-9f32-1b8c0a865126 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:22:34.032206 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:34.032206 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:34.032206 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:34.032206 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:34.032206 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:34.032206 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:34.032206 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:34.032206 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:34.032206 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:34.032206 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:34.032206 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:34.032206 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:22:34.032206 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:22:34.032206 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:22:34.033426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:22:34.033426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:34.033426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:34.033426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:34.033426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:34.033426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:34.033426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:34.033426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:34.033426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:34.033426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:34.033426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:34.033426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:34.033426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:34.033426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:34.033426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:34.033426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:34.033426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:34.034514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:22:34.034514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:22:34.034514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:22:34.034514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:22:34.034514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:22:34.034514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:22:34.034514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:22:34.034514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:22:34.040010 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 612/1244] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:22:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:22:34.040980 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ac4e6-81db-46e0-b7ee-31ccc0ad1564 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:34.164361 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ac4e6-81db-46e0-b7ee-31ccc0ad1564 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:34.223246 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e72314-13fe-4d26-8556-6e827d750416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:34.224383 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e72314-13fe-4d26-8556-6e827d750416 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:34.224796 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e72314-13fe-4d26-8556-6e827d750416 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:34.225191 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e72314-13fe-4d26-8556-6e827d750416 None admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:34.226337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e72314-13fe-4d26-8556-6e827d750416 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:34.226785 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e72314-13fe-4d26-8556-6e827d750416 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:34.229317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e72314-13fe-4d26-8556-6e827d750416 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:34.250629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 633/1245] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:22:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:34.257312 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc732d-08a8-4f61-9f32-1b8c0a865126 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:34.377868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc732d-08a8-4f61-9f32-1b8c0a865126 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:34.423603 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c11a2-643d-407a-8370-59cae0377abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:34.424205 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c11a2-643d-407a-8370-59cae0377abd None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:34.424505 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c11a2-643d-407a-8370-59cae0377abd None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:34.424897 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c11a2-643d-407a-8370-59cae0377abd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:34.425660 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c11a2-643d-407a-8370-59cae0377abd None admin] RBAC: Adding query filter params (role.id=2edb704d42834db6acd295fa3a8be5d5, scope.project.id=22768aba760b4307aab131d1312b2bf6, user.id=01c7cfbe911c4378857848966f4f8d98) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:34.426000 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c11a2-643d-407a-8370-59cae0377abd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:34.427571 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c11a2-643d-407a-8370-59cae0377abd None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:34.436333 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 613/1246] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:22:34 2022] GET /identity/v3/role_assignments?user.id=01c7cfbe911c4378857848966f4f8d98&scope.project.id=22768aba760b4307aab131d1312b2bf6&role.id=2edb704d42834db6acd295fa3a8be5d5 => generated 610 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:36.808814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd04167-6634-41f0-84b9-6d9e4c8ed88c None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:36.809478 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd04167-6634-41f0-84b9-6d9e4c8ed88c None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:36.809478 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd04167-6634-41f0-84b9-6d9e4c8ed88c None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:36.811016 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 634/1247] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:22:36 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:22:36.818340 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455bcf1-8773-4b19-a9c1-2ccabac5c506 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:36.818918 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455bcf1-8773-4b19-a9c1-2ccabac5c506 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:36.818918 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455bcf1-8773-4b19-a9c1-2ccabac5c506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:36.828221 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:36.843260 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e455bcf1-8773-4b19-a9c1-2ccabac5c506 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:22:36.844451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e455bcf1-8773-4b19-a9c1-2ccabac5c506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:22:36.934993 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 614/1248] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:22:36 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:22:36.964638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd04167-6634-41f0-84b9-6d9e4c8ed88c None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:36.989335 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:37.105433 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd04167-6634-41f0-84b9-6d9e4c8ed88c None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:37.153928 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf5729-4337-49a3-adb3-6d7c229b0a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:37.154634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf5729-4337-49a3-adb3-6d7c229b0a99 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:37.154907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf5729-4337-49a3-adb3-6d7c229b0a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:37.155185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf5729-4337-49a3-adb3-6d7c229b0a99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:37.167938 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf5729-4337-49a3-adb3-6d7c229b0a99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:22:37.174265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-94bf5729-4337-49a3-adb3-6d7c229b0a99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:22:37.174265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:37.174265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:37.174265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:37.174265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:37.174265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:37.174265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:37.174265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:37.174265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:37.174265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:37.174265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:37.174265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:37.174265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:22:37.174265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:22:37.174265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:22:37.174265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:22:37.175286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:22:37.175286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:22:37.175286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:22:37.175286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:22:37.175286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:37.175286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:37.175286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:37.175286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:37.175286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:37.175286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:37.175286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:37.175286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:37.175286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:37.175286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:37.175286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:37.175286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:37.176969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:37.176969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:37.176969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:37.176969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:37.176969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:22:37.176969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:22:37.176969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:22:37.176969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:22:37.176969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:22:37.176969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:22:37.176969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:22:37.176969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:22:37.176969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 635/1249] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:22:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:22:37.182760 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e455bcf1-8773-4b19-a9c1-2ccabac5c506 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:37.304036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e455bcf1-8773-4b19-a9c1-2ccabac5c506 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:37.339815 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:37.346454 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee34c4-75c2-4847-8a9d-82655ad0586b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:37.347104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee34c4-75c2-4847-8a9d-82655ad0586b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:37.347392 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee34c4-75c2-4847-8a9d-82655ad0586b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:37.347671 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee34c4-75c2-4847-8a9d-82655ad0586b None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:37.348511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee34c4-75c2-4847-8a9d-82655ad0586b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:37.348827 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee34c4-75c2-4847-8a9d-82655ad0586b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:37.350239 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee34c4-75c2-4847-8a9d-82655ad0586b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:37.367558 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 615/1250] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:22:37 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:37.368956 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf5729-4337-49a3-adb3-6d7c229b0a99 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:37.463410 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:37.489927 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf5729-4337-49a3-adb3-6d7c229b0a99 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:37.540786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb9992-35a7-4bfc-918c-aa52eeb5e7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:37.541782 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb9992-35a7-4bfc-918c-aa52eeb5e7af None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:37.542251 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb9992-35a7-4bfc-918c-aa52eeb5e7af None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:37.542688 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb9992-35a7-4bfc-918c-aa52eeb5e7af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:37.557368 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb9992-35a7-4bfc-918c-aa52eeb5e7af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:37.558838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb9992-35a7-4bfc-918c-aa52eeb5e7af None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:37.575438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-20eb9992-35a7-4bfc-918c-aa52eeb5e7af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:37.575438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:37.575438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:37.575438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:37.575438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:37.575438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:37.575438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:37.575438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:37.575438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:37.575438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:37.575438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:37.575438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:37.575438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:22:37.575438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:22:37.575438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:22:37.576971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:22:37.576971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:37.576971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:37.576971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:37.576971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:37.576971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:37.576971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:37.576971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:37.576971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:37.576971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:37.576971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:37.576971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:37.576971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:37.576971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:37.576971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:37.576971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:37.576971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:37.578075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:22:37.578075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:22:37.578075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:37.578075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:22:37.578075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 636/1251] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:22:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:37.584057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee34c4-75c2-4847-8a9d-82655ad0586b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:37.709486 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee34c4-75c2-4847-8a9d-82655ad0586b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:37.756339 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142e7e0-3174-4e85-9fcf-4603c405cbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:37.757014 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142e7e0-3174-4e85-9fcf-4603c405cbba None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:37.757280 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142e7e0-3174-4e85-9fcf-4603c405cbba None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:37.757636 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142e7e0-3174-4e85-9fcf-4603c405cbba None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:37.761297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142e7e0-3174-4e85-9fcf-4603c405cbba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:37.761615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142e7e0-3174-4e85-9fcf-4603c405cbba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:37.763099 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142e7e0-3174-4e85-9fcf-4603c405cbba None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:37.778546 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 616/1252] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:22:37 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:37.786734 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb9992-35a7-4bfc-918c-aa52eeb5e7af None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:37.914760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb9992-35a7-4bfc-918c-aa52eeb5e7af None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:37.950167 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:37.960678 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f28984-0bb4-4e5c-af1e-193c7032aa95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:37.961263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f28984-0bb4-4e5c-af1e-193c7032aa95 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:37.961546 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f28984-0bb4-4e5c-af1e-193c7032aa95 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:37.961824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f28984-0bb4-4e5c-af1e-193c7032aa95 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:37.972332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f28984-0bb4-4e5c-af1e-193c7032aa95 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:37.973835 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f28984-0bb4-4e5c-af1e-193c7032aa95 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:37.988100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-a6f28984-0bb4-4e5c-af1e-193c7032aa95 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 10:22:37.988100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:37.988100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:37.988100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:37.988100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:37.988100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:37.988100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:37.988100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:37.988100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:37.988100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:37.988100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:37.988100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:37.988100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:22:37.988100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:22:37.988100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:22:37.988100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:22:37.990150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:37.990150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:37.990150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:22:37.990150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:22:37.990150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:22:37.990150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:22:37.990150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:37.990150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:37.990150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:37.990150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:37.990150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:37.990150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:37.990150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:37.990150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:37.990150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:37.990150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:37.990150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:37.990150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:37.992669 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:37.992669 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:37.992669 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:22:37.992669 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:22:37.992669 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:22:37.992669 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:22:37.992669 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:22:37.992669 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:22:37.992669 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 10:22:37.992669 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:22:37.992669 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 637/1253] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:22:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:37.998253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142e7e0-3174-4e85-9fcf-4603c405cbba None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:38.086328 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:38.177942 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142e7e0-3174-4e85-9fcf-4603c405cbba None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:38.239509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b7d59-be1f-42bd-8896-841120ce2d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:38.240145 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b7d59-be1f-42bd-8896-841120ce2d78 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:38.240433 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b7d59-be1f-42bd-8896-841120ce2d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:38.240956 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b7d59-be1f-42bd-8896-841120ce2d78 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:38.243882 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b7d59-be1f-42bd-8896-841120ce2d78 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:38.244247 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b7d59-be1f-42bd-8896-841120ce2d78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:38.245913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b7d59-be1f-42bd-8896-841120ce2d78 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:38.269939 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 617/1254] 10.222.0.121 () {56 vars in 1148 bytes} [Thu Aug 4 10:22:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:22:38.276129 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f28984-0bb4-4e5c-af1e-193c7032aa95 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:38.419552 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f28984-0bb4-4e5c-af1e-193c7032aa95 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:38.475553 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9e883-4b30-4f24-a6f4-f1808bd1ccb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:38.478762 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9e883-4b30-4f24-a6f4-f1808bd1ccb7 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:38.478762 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9e883-4b30-4f24-a6f4-f1808bd1ccb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:38.478762 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9e883-4b30-4f24-a6f4-f1808bd1ccb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:38.565133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:38.581924 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab9e883-4b30-4f24-a6f4-f1808bd1ccb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:38.583682 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab9e883-4b30-4f24-a6f4-f1808bd1ccb7 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:38.722342 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 638/1255] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:22:38 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:22:38.730897 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b7d59-be1f-42bd-8896-841120ce2d78 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:38.876830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b7d59-be1f-42bd-8896-841120ce2d78 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:38.894428 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:38.926690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59b885-d71f-4016-a3fe-b938eff61aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:38.927406 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59b885-d71f-4016-a3fe-b938eff61aeb None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:38.927691 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59b885-d71f-4016-a3fe-b938eff61aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:38.928064 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59b885-d71f-4016-a3fe-b938eff61aeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:38.947894 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d59b885-d71f-4016-a3fe-b938eff61aeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:38.950280 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d59b885-d71f-4016-a3fe-b938eff61aeb None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:38.971135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-4d59b885-d71f-4016-a3fe-b938eff61aeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:38.971135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:38.971135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:38.971135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:38.971135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:38.971135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:38.971135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:38.971135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:38.971135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:38.971135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:38.971135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:38.971135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:38.971135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:22:38.971135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:22:38.971135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:22:38.972673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:22:38.972673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:38.972673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:38.972673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:38.972673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:38.972673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:38.972673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:38.972673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:38.972673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:38.972673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:38.972673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:38.972673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:38.972673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:38.972673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:38.972673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:38.972673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:38.972673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:38.973844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:22:38.973844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:22:38.973844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:38.973844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:22:38.973844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 618/1256] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:22:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:38.981753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9e883-4b30-4f24-a6f4-f1808bd1ccb7 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:39.116947 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:39.135170 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9e883-4b30-4f24-a6f4-f1808bd1ccb7 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:39.188292 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21080b-21f7-4a63-b2f8-c58f3e6a3150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:39.189166 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21080b-21f7-4a63-b2f8-c58f3e6a3150 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:39.189484 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21080b-21f7-4a63-b2f8-c58f3e6a3150 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:39.189798 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21080b-21f7-4a63-b2f8-c58f3e6a3150 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:39.190805 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d21080b-21f7-4a63-b2f8-c58f3e6a3150 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:39.191145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d21080b-21f7-4a63-b2f8-c58f3e6a3150 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:39.192839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d21080b-21f7-4a63-b2f8-c58f3e6a3150 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:39.208110 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 639/1257] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:22:38 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:22:39.214415 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59b885-d71f-4016-a3fe-b938eff61aeb None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:39.338626 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59b885-d71f-4016-a3fe-b938eff61aeb None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:39.387356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b426bde-b083-44db-a8ec-8935dc6933ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:39.387933 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b426bde-b083-44db-a8ec-8935dc6933ab None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:39.388257 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b426bde-b083-44db-a8ec-8935dc6933ab None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:39.388532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b426bde-b083-44db-a8ec-8935dc6933ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:39.397454 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b426bde-b083-44db-a8ec-8935dc6933ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:39.398941 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b426bde-b083-44db-a8ec-8935dc6933ab None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:39.411656 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-4b426bde-b083-44db-a8ec-8935dc6933ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:22:39.411656 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:39.411656 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:39.411656 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:39.411656 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:39.411656 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:39.411656 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:39.411656 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:39.411656 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:39.411656 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:39.411656 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:39.411656 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:39.411656 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:22:39.411656 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:22:39.411656 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:22:39.413444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:22:39.413444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:39.413444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:39.413444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:39.413444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:39.413444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:39.413444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:39.413444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:39.413444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:39.413444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:39.413444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:39.413444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:39.413444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:39.413444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:39.413444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:39.413444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:39.413444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:39.415166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:22:39.415166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:22:39.415166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:22:39.415166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:22:39.415166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:22:39.415166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:22:39.415166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:22:39.415166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:22:39.415166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 619/1258] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:22:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 10:22:39.420008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21080b-21f7-4a63-b2f8-c58f3e6a3150 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:39.559108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21080b-21f7-4a63-b2f8-c58f3e6a3150 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:39.616500 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330522b4-96a7-426c-a2c8-29a93bf5ba51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:39.617545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330522b4-96a7-426c-a2c8-29a93bf5ba51 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:39.617970 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330522b4-96a7-426c-a2c8-29a93bf5ba51 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:39.618492 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330522b4-96a7-426c-a2c8-29a93bf5ba51 None admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:39.619627 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330522b4-96a7-426c-a2c8-29a93bf5ba51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:39.620087 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330522b4-96a7-426c-a2c8-29a93bf5ba51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:39.621971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330522b4-96a7-426c-a2c8-29a93bf5ba51 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:39.639698 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 640/1259] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:22:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:39.646325 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b426bde-b083-44db-a8ec-8935dc6933ab None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:39.681674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:39.783826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b426bde-b083-44db-a8ec-8935dc6933ab None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:39.822845 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ea136-6e96-49a6-9b97-db5deaf3b587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:39.823491 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ea136-6e96-49a6-9b97-db5deaf3b587 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:39.823775 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ea136-6e96-49a6-9b97-db5deaf3b587 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:39.824090 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ea136-6e96-49a6-9b97-db5deaf3b587 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:39.824904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ea136-6e96-49a6-9b97-db5deaf3b587 None admin] RBAC: Adding query filter params (role.id=7342106a255b4bce8f150726602a8b95, scope.project.id=22768aba760b4307aab131d1312b2bf6, user.id=01c7cfbe911c4378857848966f4f8d98) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:39.825217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ea136-6e96-49a6-9b97-db5deaf3b587 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:39.826738 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ea136-6e96-49a6-9b97-db5deaf3b587 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:39.833435 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 620/1260] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:22:39 2022] GET /identity/v3/role_assignments?user.id=01c7cfbe911c4378857848966f4f8d98&scope.project.id=22768aba760b4307aab131d1312b2bf6&role.id=7342106a255b4bce8f150726602a8b95 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:22:42.069113 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2d19c-7fdd-4953-acab-919ddeb9cf38 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:42.069851 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2d19c-7fdd-4953-acab-919ddeb9cf38 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:42.069851 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2d19c-7fdd-4953-acab-919ddeb9cf38 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:42.071296 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 641/1261] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:22:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:22:42.077286 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193544a-79e3-4b8c-be57-aefb21dd37a8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:42.077915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193544a-79e3-4b8c-be57-aefb21dd37a8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:42.077915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193544a-79e3-4b8c-be57-aefb21dd37a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:42.102712 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8193544a-79e3-4b8c-be57-aefb21dd37a8 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:22:42.107299 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8193544a-79e3-4b8c-be57-aefb21dd37a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:22:42.191820 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 621/1262] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:22:42 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:22:42.215743 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2d19c-7fdd-4953-acab-919ddeb9cf38 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:42.228436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:42.330335 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2d19c-7fdd-4953-acab-919ddeb9cf38 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:42.369703 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee48fa-3a5d-49d7-9e7e-6fc76992434b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:42.370371 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee48fa-3a5d-49d7-9e7e-6fc76992434b None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:42.370673 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee48fa-3a5d-49d7-9e7e-6fc76992434b None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:42.370939 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee48fa-3a5d-49d7-9e7e-6fc76992434b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:42.382535 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee48fa-3a5d-49d7-9e7e-6fc76992434b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:22:42.387588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-3cee48fa-3a5d-49d7-9e7e-6fc76992434b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:22:42.387588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:42.387588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:42.387588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:42.387588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:42.387588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:42.387588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:42.387588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:42.387588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:42.387588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:42.387588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:42.387588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:42.387588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:22:42.387588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:22:42.387588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:22:42.387588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:22:42.388659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:22:42.388659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:22:42.388659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:22:42.388659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:22:42.388659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:42.388659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:42.388659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:42.388659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:42.388659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:42.388659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:42.388659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:42.388659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:42.388659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:42.388659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:42.388659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:42.388659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:42.390238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:42.390238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:42.390238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:42.390238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:42.390238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:22:42.390238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:22:42.390238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:22:42.390238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:22:42.390238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:22:42.390238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:22:42.390238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:22:42.390238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:22:42.390238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 642/1263] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:22:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:42.395271 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193544a-79e3-4b8c-be57-aefb21dd37a8 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:42.420343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:42.510027 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193544a-79e3-4b8c-be57-aefb21dd37a8 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:42.545546 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06d87d-058d-47c1-a472-92e64a65cdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:42.546186 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06d87d-058d-47c1-a472-92e64a65cdc2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:42.546467 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06d87d-058d-47c1-a472-92e64a65cdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:42.546738 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06d87d-058d-47c1-a472-92e64a65cdc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:42.547462 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06d87d-058d-47c1-a472-92e64a65cdc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:42.547765 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06d87d-058d-47c1-a472-92e64a65cdc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:42.549181 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06d87d-058d-47c1-a472-92e64a65cdc2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:42.558056 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 622/1264] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:22:42 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:42.564352 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee48fa-3a5d-49d7-9e7e-6fc76992434b None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:42.636387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:42.675377 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee48fa-3a5d-49d7-9e7e-6fc76992434b None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:42.714370 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea7ce8-5753-4e08-af8a-592c8cf9f251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:42.715035 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea7ce8-5753-4e08-af8a-592c8cf9f251 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:42.715323 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea7ce8-5753-4e08-af8a-592c8cf9f251 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:42.715592 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea7ce8-5753-4e08-af8a-592c8cf9f251 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:42.728605 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea7ce8-5753-4e08-af8a-592c8cf9f251 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:42.730092 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea7ce8-5753-4e08-af8a-592c8cf9f251 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:42.745061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-59ea7ce8-5753-4e08-af8a-592c8cf9f251 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:42.745061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:42.745061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:42.745061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:42.745061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:42.745061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:42.745061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:42.745061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:42.745061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:42.745061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:42.745061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:42.745061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:42.745061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:22:42.745061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:22:42.745061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:22:42.746046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:22:42.746046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:42.746046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:42.746046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:42.746046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:42.746046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:42.746046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:42.746046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:42.746046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:42.746046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:42.746046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:42.746046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:42.746046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:42.746046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:42.746046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:42.746046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:42.746046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:42.747025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:22:42.747025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:22:42.747025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:42.747025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:22:42.747025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 643/1265] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:22:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:42.752678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06d87d-058d-47c1-a472-92e64a65cdc2 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:42.840096 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:42.861609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06d87d-058d-47c1-a472-92e64a65cdc2 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:42.898093 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35da932-28df-41d4-8d86-fae825b25c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:42.898726 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35da932-28df-41d4-8d86-fae825b25c95 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:42.899006 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35da932-28df-41d4-8d86-fae825b25c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:42.899279 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35da932-28df-41d4-8d86-fae825b25c95 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:42.900010 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35da932-28df-41d4-8d86-fae825b25c95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:42.900320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35da932-28df-41d4-8d86-fae825b25c95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:42.901716 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35da932-28df-41d4-8d86-fae825b25c95 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:42.915255 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 623/1266] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:22:42 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:42.921616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea7ce8-5753-4e08-af8a-592c8cf9f251 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:43.026463 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea7ce8-5753-4e08-af8a-592c8cf9f251 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:43.046822 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:43.063533 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ca99d-1411-4338-a085-4bac6a087781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:43.064234 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ca99d-1411-4338-a085-4bac6a087781 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:43.064518 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ca99d-1411-4338-a085-4bac6a087781 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:43.064795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ca99d-1411-4338-a085-4bac6a087781 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:43.072715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62ca99d-1411-4338-a085-4bac6a087781 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:43.074201 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62ca99d-1411-4338-a085-4bac6a087781 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:43.085557 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-e62ca99d-1411-4338-a085-4bac6a087781 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 10:22:43.085557 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:43.085557 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:43.085557 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:43.085557 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:43.085557 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:43.085557 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:43.085557 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:43.085557 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:43.085557 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:43.085557 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:43.085557 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:43.085557 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:22:43.085557 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:22:43.085557 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:22:43.085557 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:22:43.086646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:43.086646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:43.086646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:22:43.086646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:22:43.086646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:22:43.086646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:22:43.086646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:43.086646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:43.086646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:43.086646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:43.086646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:43.086646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:43.086646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:43.086646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:43.086646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:43.086646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:43.086646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:43.086646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:43.087642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:43.087642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:43.087642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:22:43.087642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:22:43.087642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:22:43.087642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:22:43.087642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:22:43.087642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:22:43.087642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 10:22:43.087642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:22:43.087642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 644/1267] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:22:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:43.093060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35da932-28df-41d4-8d86-fae825b25c95 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:43.205004 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35da932-28df-41d4-8d86-fae825b25c95 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:43.242073 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc940c67-fe4a-4466-bfec-f180d6028e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:43.242676 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc940c67-fe4a-4466-bfec-f180d6028e48 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:43.242957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc940c67-fe4a-4466-bfec-f180d6028e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:43.243237 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc940c67-fe4a-4466-bfec-f180d6028e48 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:43.244096 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc940c67-fe4a-4466-bfec-f180d6028e48 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:43.244420 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc940c67-fe4a-4466-bfec-f180d6028e48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:43.245862 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc940c67-fe4a-4466-bfec-f180d6028e48 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:43.255406 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:43.264477 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 624/1268] 10.222.0.121 () {56 vars in 1148 bytes} [Thu Aug 4 10:22:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:43.270564 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ca99d-1411-4338-a085-4bac6a087781 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:43.376852 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ca99d-1411-4338-a085-4bac6a087781 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:43.412875 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688b0f3-939d-4746-84fc-6fcc0b828993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:43.413535 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688b0f3-939d-4746-84fc-6fcc0b828993 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:43.413820 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688b0f3-939d-4746-84fc-6fcc0b828993 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:43.414105 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688b0f3-939d-4746-84fc-6fcc0b828993 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:43.425682 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9688b0f3-939d-4746-84fc-6fcc0b828993 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:43.427209 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9688b0f3-939d-4746-84fc-6fcc0b828993 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:43.442892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-9688b0f3-939d-4746-84fc-6fcc0b828993 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:43.442892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:43.442892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:43.442892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:43.442892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:43.442892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:43.442892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:43.442892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:43.442892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:43.442892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:43.442892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:43.442892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:43.442892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:22:43.442892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:22:43.442892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:22:43.444168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:22:43.444168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:43.444168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:43.444168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:43.444168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:43.444168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:43.444168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:43.444168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:43.444168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:43.444168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:43.444168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:43.444168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:43.444168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:43.444168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:43.444168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:43.444168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:43.444168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:43.445238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:22:43.445238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:22:43.445238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:43.445238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:22:43.445238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 645/1269] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:22:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:43.450530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc940c67-fe4a-4466-bfec-f180d6028e48 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:43.562655 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc940c67-fe4a-4466-bfec-f180d6028e48 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:43.603652 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13143a84-1c53-42a9-ba6f-a279bbb0392e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:43.604307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13143a84-1c53-42a9-ba6f-a279bbb0392e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:43.604583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13143a84-1c53-42a9-ba6f-a279bbb0392e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:43.604868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13143a84-1c53-42a9-ba6f-a279bbb0392e None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:43.605575 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13143a84-1c53-42a9-ba6f-a279bbb0392e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:43.605887 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13143a84-1c53-42a9-ba6f-a279bbb0392e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:43.607303 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13143a84-1c53-42a9-ba6f-a279bbb0392e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:43.620680 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 625/1270] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:22:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:43.627222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688b0f3-939d-4746-84fc-6fcc0b828993 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:43.664906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:43.740499 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688b0f3-939d-4746-84fc-6fcc0b828993 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:43.778088 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a2776-f05a-43ef-bd62-5de680ffffa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:43.778784 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a2776-f05a-43ef-bd62-5de680ffffa1 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:43.779046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a2776-f05a-43ef-bd62-5de680ffffa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:43.779325 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a2776-f05a-43ef-bd62-5de680ffffa1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:43.786685 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a2776-f05a-43ef-bd62-5de680ffffa1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:43.788223 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a2776-f05a-43ef-bd62-5de680ffffa1 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:43.800634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-388a2776-f05a-43ef-bd62-5de680ffffa1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:22:43.800634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:43.800634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:43.800634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:43.800634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:43.800634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:43.800634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:43.800634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:43.800634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:43.800634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:43.800634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:43.800634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:43.800634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:22:43.800634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:22:43.800634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:22:43.801819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:22:43.801819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:43.801819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:43.801819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:43.801819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:43.801819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:43.801819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:43.801819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:43.801819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:43.801819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:43.801819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:43.801819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:43.801819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:43.801819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:43.801819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:43.801819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:43.801819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:43.802927 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:22:43.802927 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:22:43.802927 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:22:43.802927 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:22:43.802927 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:22:43.802927 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:22:43.802927 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:22:43.802927 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:22:43.802927 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 646/1271] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:22:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:43.808374 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13143a84-1c53-42a9-ba6f-a279bbb0392e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:43.874102 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:43.920770 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13143a84-1c53-42a9-ba6f-a279bbb0392e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:43.954899 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8593fc-eba3-4ab5-8a11-00bbfd2770d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:43.955525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8593fc-eba3-4ab5-8a11-00bbfd2770d8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:43.955806 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8593fc-eba3-4ab5-8a11-00bbfd2770d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:43.956114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8593fc-eba3-4ab5-8a11-00bbfd2770d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:43.956869 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8593fc-eba3-4ab5-8a11-00bbfd2770d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:43.957179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8593fc-eba3-4ab5-8a11-00bbfd2770d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:43.958734 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8593fc-eba3-4ab5-8a11-00bbfd2770d8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:43.971470 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 626/1272] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:22:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:43.978058 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a2776-f05a-43ef-bd62-5de680ffffa1 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:44.078800 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:44.086915 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a2776-f05a-43ef-bd62-5de680ffffa1 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:44.126747 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f98cf-7884-4d61-a789-e8d47276e43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:44.127437 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f98cf-7884-4d61-a789-e8d47276e43d None admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:44.127719 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f98cf-7884-4d61-a789-e8d47276e43d None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:44.128041 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f98cf-7884-4d61-a789-e8d47276e43d None admin] PATH_INFO: `/v3/projects/22768aba760b4307aab131d1312b2bf6/users/01c7cfbe911c4378857848966f4f8d98/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:44.130225 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f98cf-7884-4d61-a789-e8d47276e43d None admin] RBAC: Authorizing `identity:create_grant(project_id=22768aba760b4307aab131d1312b2bf6, user_id=01c7cfbe911c4378857848966f4f8d98, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:44.131832 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f98cf-7884-4d61-a789-e8d47276e43d None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:44.145465 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 647/1273] 10.222.0.121 () {58 vars in 1444 bytes} [Thu Aug 4 10:22:43 2022] PUT /identity/v3/projects/22768aba760b4307aab131d1312b2bf6/users/01c7cfbe911c4378857848966f4f8d98/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:22:46.178328 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc2fed-0cf6-4fb3-a716-24aecbf99dbe None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:46.179024 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc2fed-0cf6-4fb3-a716-24aecbf99dbe None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:46.179024 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc2fed-0cf6-4fb3-a716-24aecbf99dbe None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:46.180325 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 627/1274] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:22:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:22:46.186850 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b65f2b-cdd6-4ade-a6c0-2b13f74429e3 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:46.187582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b65f2b-cdd6-4ade-a6c0-2b13f74429e3 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:46.187742 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b65f2b-cdd6-4ade-a6c0-2b13f74429e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:46.207332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-a0b65f2b-cdd6-4ade-a6c0-2b13f74429e3 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:22:46.209088 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-a0b65f2b-cdd6-4ade-a6c0-2b13f74429e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:22:46.280270 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 648/1275] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:22:46 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:22:46.305818 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc2fed-0cf6-4fb3-a716-24aecbf99dbe None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:46.422478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc2fed-0cf6-4fb3-a716-24aecbf99dbe None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:46.440830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:46.459874 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a5506-e700-4120-8ef1-609eee8a9f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:46.460564 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a5506-e700-4120-8ef1-609eee8a9f71 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:46.460834 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a5506-e700-4120-8ef1-609eee8a9f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:46.461124 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a5506-e700-4120-8ef1-609eee8a9f71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:46.474196 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a5506-e700-4120-8ef1-609eee8a9f71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 10:22:46.479514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-031a5506-e700-4120-8ef1-609eee8a9f71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:22:46.479514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:46.479514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:46.479514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:46.479514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:46.479514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:46.479514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:46.479514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:46.479514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:46.479514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:46.479514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:46.479514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:46.479514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 10:22:46.479514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 10:22:46.479514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 10:22:46.479514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 10:22:46.480803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 10:22:46.480803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 10:22:46.480803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 10:22:46.480803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 10:22:46.480803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:46.480803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:46.480803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:46.480803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:46.480803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:46.480803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:46.480803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:46.480803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:46.480803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:46.480803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:46.480803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:46.480803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:46.481879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:46.481879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:46.481879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:46.481879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:46.481879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 10:22:46.481879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 10:22:46.481879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 10:22:46.481879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 10:22:46.481879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 10:22:46.481879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 10:22:46.481879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 10:22:46.481879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:22:46.481879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 628/1276] 10.222.0.121 () {56 vars in 1101 bytes} [Thu Aug 4 10:22:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:46.487720 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b65f2b-cdd6-4ade-a6c0-2b13f74429e3 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:46.571007 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:46.590687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b65f2b-cdd6-4ade-a6c0-2b13f74429e3 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:46.627348 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1083f-db0c-4c06-b42d-0474094f7d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:46.627981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1083f-db0c-4c06-b42d-0474094f7d40 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:46.628292 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1083f-db0c-4c06-b42d-0474094f7d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:46.628566 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1083f-db0c-4c06-b42d-0474094f7d40 None admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:46.629298 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1083f-db0c-4c06-b42d-0474094f7d40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:46.629665 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1083f-db0c-4c06-b42d-0474094f7d40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:46.631075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1083f-db0c-4c06-b42d-0474094f7d40 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:46.639167 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 649/1277] 10.222.0.121 () {56 vars in 1104 bytes} [Thu Aug 4 10:22:46 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:46.645749 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a5506-e700-4120-8ef1-609eee8a9f71 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:46.751578 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a5506-e700-4120-8ef1-609eee8a9f71 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:46.786671 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406270b-6daf-4ac9-b0f2-7df79f308ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:46.787316 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406270b-6daf-4ac9-b0f2-7df79f308ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:46.787559 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406270b-6daf-4ac9-b0f2-7df79f308ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:46.787830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406270b-6daf-4ac9-b0f2-7df79f308ae0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:46.798470 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0406270b-6daf-4ac9-b0f2-7df79f308ae0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:46.799975 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0406270b-6daf-4ac9-b0f2-7df79f308ae0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:46.815343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-0406270b-6daf-4ac9-b0f2-7df79f308ae0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:46.815343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:46.815343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:46.815343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:46.815343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:46.815343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:46.815343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:46.815343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:46.815343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:46.815343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:46.815343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:46.815343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:46.815343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:22:46.815343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:22:46.815343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:22:46.816525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:22:46.816525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:46.816525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:46.816525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:46.816525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:46.816525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:46.816525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:46.816525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:46.816525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:46.816525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:46.816525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:46.816525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:46.816525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:46.816525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:46.816525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:46.816525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:46.816525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:46.817497 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:22:46.817497 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:22:46.817497 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:46.817497 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:22:46.817497 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 629/1278] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:22:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:46.822857 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1083f-db0c-4c06-b42d-0474094f7d40 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:46.930218 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1083f-db0c-4c06-b42d-0474094f7d40 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:46.966266 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb59ce-5361-49c2-a310-2a5652185d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:46.966905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb59ce-5361-49c2-a310-2a5652185d24 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:46.967194 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb59ce-5361-49c2-a310-2a5652185d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:46.967472 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb59ce-5361-49c2-a310-2a5652185d24 None admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:46.968218 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb59ce-5361-49c2-a310-2a5652185d24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:46.968524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb59ce-5361-49c2-a310-2a5652185d24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:46.969941 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb59ce-5361-49c2-a310-2a5652185d24 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:46.981683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 650/1279] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:22:46 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:46.988261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406270b-6daf-4ac9-b0f2-7df79f308ae0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:47.023237 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:47.102053 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406270b-6daf-4ac9-b0f2-7df79f308ae0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:47.141651 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc87c9d1-6269-4748-a64c-7e4d3a97ae56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:47.142457 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc87c9d1-6269-4748-a64c-7e4d3a97ae56 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:47.142738 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc87c9d1-6269-4748-a64c-7e4d3a97ae56 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:47.143075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc87c9d1-6269-4748-a64c-7e4d3a97ae56 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:47.153542 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc87c9d1-6269-4748-a64c-7e4d3a97ae56 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:47.155369 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc87c9d1-6269-4748-a64c-7e4d3a97ae56 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:47.167978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-fc87c9d1-6269-4748-a64c-7e4d3a97ae56 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 10:22:47.167978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:47.167978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:47.167978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:47.167978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:47.167978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:47.167978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:47.167978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:47.167978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:47.167978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:47.167978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:47.167978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:47.167978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 10:22:47.167978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 10:22:47.167978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 10:22:47.167978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 10:22:47.169320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:47.169320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:47.169320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 10:22:47.169320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:22:47.169320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 10:22:47.169320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 10:22:47.169320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:47.169320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:47.169320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:47.169320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:47.169320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:47.169320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:47.169320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:47.169320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:47.169320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:47.169320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:47.169320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:47.169320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:47.170314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:47.170314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:47.170314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 10:22:47.170314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 10:22:47.170314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 10:22:47.170314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 10:22:47.170314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 10:22:47.170314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 10:22:47.170314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 10:22:47.170314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:22:47.170314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 630/1280] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:22:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:47.175625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb59ce-5361-49c2-a310-2a5652185d24 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:47.179806 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:47.288273 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb59ce-5361-49c2-a310-2a5652185d24 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:47.322467 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17038862-ad9c-48b6-98c1-5c894e2c51c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:47.323111 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17038862-ad9c-48b6-98c1-5c894e2c51c5 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:47.323394 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17038862-ad9c-48b6-98c1-5c894e2c51c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:47.323675 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17038862-ad9c-48b6-98c1-5c894e2c51c5 None admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:47.324513 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17038862-ad9c-48b6-98c1-5c894e2c51c5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:47.324840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17038862-ad9c-48b6-98c1-5c894e2c51c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:47.326334 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17038862-ad9c-48b6-98c1-5c894e2c51c5 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:47.345579 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 651/1281] 10.222.0.121 () {56 vars in 1148 bytes} [Thu Aug 4 10:22:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:47.352830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc87c9d1-6269-4748-a64c-7e4d3a97ae56 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:47.459173 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:47.470715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc87c9d1-6269-4748-a64c-7e4d3a97ae56 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:47.511747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5f039-554a-413d-a39c-18d5a7323fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:47.512415 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5f039-554a-413d-a39c-18d5a7323fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:47.512704 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5f039-554a-413d-a39c-18d5a7323fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:47.512984 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5f039-554a-413d-a39c-18d5a7323fe7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:47.583881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5f039-554a-413d-a39c-18d5a7323fe7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:47.584983 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5f039-554a-413d-a39c-18d5a7323fe7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:47.676343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:47.695696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 631/1282] 10.222.0.121 () {58 vars in 1287 bytes} [Thu Aug 4 10:22:47 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 10:22:47.701984 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17038862-ad9c-48b6-98c1-5c894e2c51c5 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:47.766262 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:47.810188 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17038862-ad9c-48b6-98c1-5c894e2c51c5 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:47.847324 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17e2fb-ef77-4272-bd78-c64e562d32d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:47.847894 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17e2fb-ef77-4272-bd78-c64e562d32d7 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:47.848216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17e2fb-ef77-4272-bd78-c64e562d32d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:47.848531 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17e2fb-ef77-4272-bd78-c64e562d32d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:47.859332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17e2fb-ef77-4272-bd78-c64e562d32d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:47.860847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17e2fb-ef77-4272-bd78-c64e562d32d7 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:47.876422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-4b17e2fb-ef77-4272-bd78-c64e562d32d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:47.876422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:47.876422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:47.876422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:47.876422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:47.876422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:47.876422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:47.876422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:47.876422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:47.876422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:47.876422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:47.876422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:47.876422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 10:22:47.876422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 10:22:47.876422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 10:22:47.877501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 10:22:47.877501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:47.877501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:47.877501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:47.877501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:47.877501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:47.877501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:47.877501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:47.877501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:47.877501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:47.877501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:47.877501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:47.877501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:47.877501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:47.877501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:47.877501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:47.877501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:47.878479 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 10:22:47.878479 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 10:22:47.878479 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 10:22:47.878479 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:22:47.878479 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 652/1283] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:22:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:47.883974 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5f039-554a-413d-a39c-18d5a7323fe7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:47.992319 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5f039-554a-413d-a39c-18d5a7323fe7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:48.027915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4facb9b5-49e7-46c3-92e6-f45f31da1b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:48.028576 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4facb9b5-49e7-46c3-92e6-f45f31da1b19 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:48.028850 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4facb9b5-49e7-46c3-92e6-f45f31da1b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:48.029104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4facb9b5-49e7-46c3-92e6-f45f31da1b19 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:48.029807 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4facb9b5-49e7-46c3-92e6-f45f31da1b19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:48.030119 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4facb9b5-49e7-46c3-92e6-f45f31da1b19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:48.031517 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4facb9b5-49e7-46c3-92e6-f45f31da1b19 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:48.044198 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 632/1284] 10.222.0.121 () {56 vars in 1114 bytes} [Thu Aug 4 10:22:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:48.050412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17e2fb-ef77-4272-bd78-c64e562d32d7 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:48.166256 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17e2fb-ef77-4272-bd78-c64e562d32d7 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:48.180897 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:48.204394 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ce4cc-b706-4b21-84ad-0d3eafd84e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:48.205061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ce4cc-b706-4b21-84ad-0d3eafd84e63 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:48.205350 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ce4cc-b706-4b21-84ad-0d3eafd84e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:48.205687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ce4cc-b706-4b21-84ad-0d3eafd84e63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:48.213204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ce4cc-b706-4b21-84ad-0d3eafd84e63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:48.214686 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ce4cc-b706-4b21-84ad-0d3eafd84e63 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:48.226691 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application [None req-6c9ce4cc-b706-4b21-84ad-0d3eafd84e63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:22:48.226691 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:48.226691 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:48.226691 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:48.226691 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:48.226691 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:48.226691 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:48.226691 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:48.226691 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:48.226691 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:48.226691 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:48.226691 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:48.226691 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 10:22:48.226691 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 10:22:48.226691 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 10:22:48.227840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 10:22:48.227840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:48.227840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:48.227840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:48.227840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:48.227840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:48.227840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:48.227840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:48.227840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:48.227840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:48.227840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:48.227840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:48.227840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:48.227840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:48.227840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:48.227840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:48.227840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:48.229043 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 10:22:48.229043 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 10:22:48.229043 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 10:22:48.229043 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 10:22:48.229043 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 10:22:48.229043 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 10:22:48.229043 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 10:22:48.229043 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: ERROR keystone.server.flask.application Aug 04 10:22:48.229043 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 653/1285] 10.222.0.121 () {56 vars in 1116 bytes} [Thu Aug 4 10:22:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:48.234186 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4facb9b5-49e7-46c3-92e6-f45f31da1b19 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:48.291626 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:48.346698 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4facb9b5-49e7-46c3-92e6-f45f31da1b19 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:48.384292 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de184db9-788a-4d64-9696-0342eaf9da69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:48.385088 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de184db9-788a-4d64-9696-0342eaf9da69 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:48.385184 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de184db9-788a-4d64-9696-0342eaf9da69 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:48.385487 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de184db9-788a-4d64-9696-0342eaf9da69 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:48.386253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de184db9-788a-4d64-9696-0342eaf9da69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:48.386569 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de184db9-788a-4d64-9696-0342eaf9da69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:48.388106 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de184db9-788a-4d64-9696-0342eaf9da69 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:48.400628 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 633/1286] 10.222.0.121 () {56 vars in 1153 bytes} [Thu Aug 4 10:22:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:48.407363 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ce4cc-b706-4b21-84ad-0d3eafd84e63 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:48.527927 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ce4cc-b706-4b21-84ad-0d3eafd84e63 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:48.565506 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483dacba-3381-4083-9b58-c4f37d6f6b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:48.566150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483dacba-3381-4083-9b58-c4f37d6f6b1a None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:48.566440 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483dacba-3381-4083-9b58-c4f37d6f6b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:48.566719 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483dacba-3381-4083-9b58-c4f37d6f6b1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:48.567624 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483dacba-3381-4083-9b58-c4f37d6f6b1a None admin] RBAC: Adding query filter params (role.id=7342106a255b4bce8f150726602a8b95, scope.project.id=22768aba760b4307aab131d1312b2bf6, user.id=01c7cfbe911c4378857848966f4f8d98) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:48.567937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483dacba-3381-4083-9b58-c4f37d6f6b1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:48.569444 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483dacba-3381-4083-9b58-c4f37d6f6b1a None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:48.575851 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 654/1287] 10.222.0.121 () {56 vars in 1379 bytes} [Thu Aug 4 10:22:48 2022] GET /identity/v3/role_assignments?user.id=01c7cfbe911c4378857848966f4f8d98&scope.project.id=22768aba760b4307aab131d1312b2bf6&role.id=7342106a255b4bce8f150726602a8b95 => generated 610 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:50.678666 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86bc9c-6fe2-4a60-ab09-4d1451d09412 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:50.679374 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86bc9c-6fe2-4a60-ab09-4d1451d09412 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:50.679374 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86bc9c-6fe2-4a60-ab09-4d1451d09412 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:50.680635 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 634/1288] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:22:50 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:22:50.686069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04abf8-d836-4aa2-ab22-aedb3b88317c None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:50.686596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04abf8-d836-4aa2-ab22-aedb3b88317c None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:50.686596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04abf8-d836-4aa2-ab22-aedb3b88317c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:50.712755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-7f04abf8-d836-4aa2-ab22-aedb3b88317c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:22:50.714149 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-7f04abf8-d836-4aa2-ab22-aedb3b88317c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:22:50.747303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:50.789944 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 655/1289] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:22:50 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:22:50.813371 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86bc9c-6fe2-4a60-ab09-4d1451d09412 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:50.923504 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86bc9c-6fe2-4a60-ab09-4d1451d09412 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:50.935474 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:50.961423 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a1d9e-7a12-46ac-b8b5-c93d37e8ff1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:50.962088 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a1d9e-7a12-46ac-b8b5-c93d37e8ff1a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:50.962363 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a1d9e-7a12-46ac-b8b5-c93d37e8ff1a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:50.962636 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a1d9e-7a12-46ac-b8b5-c93d37e8ff1a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:50.963285 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a1d9e-7a12-46ac-b8b5-c93d37e8ff1a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:50.964711 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a1d9e-7a12-46ac-b8b5-c93d37e8ff1a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:50.974251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-8f2a1d9e-7a12-46ac-b8b5-c93d37e8ff1a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 10:22:50.974251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:50.974251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:22:50.974251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:22:50.974251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:22:50.974251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:22:50.974251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:22:50.974251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:22:50.974251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 10:22:50.974251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:22:50.974251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:22:50.974251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:22:50.974251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:50.974251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:50.974251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:50.974251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:50.974251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:50.975481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:50.975481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:50.975481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:50.975481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:50.975481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:50.975481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:50.975481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:22:50.975481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:22:50.975481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:22:50.975481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:22:50.975481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:50.975481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:50.975481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:50.975481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:50.975481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:50.975481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:50.976689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:50.976689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:50.976689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:50.976689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:50.976689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:50.976689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:50.976689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:50.976689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:50.976689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:50.976689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:50.976689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:22:50.976689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:22:50.976689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 10:22:50.976689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:22:50.976689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 635/1290] 10.222.0.121 () {56 vars in 1122 bytes} [Thu Aug 4 10:22:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:50.982139 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04abf8-d836-4aa2-ab22-aedb3b88317c None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:51.086038 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04abf8-d836-4aa2-ab22-aedb3b88317c None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:51.128418 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f951b-4ba3-4e35-b1f4-9008ae8796c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:51.129062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f951b-4ba3-4e35-b1f4-9008ae8796c5 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:51.129428 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f951b-4ba3-4e35-b1f4-9008ae8796c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:51.129802 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f951b-4ba3-4e35-b1f4-9008ae8796c5 None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:51.130562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f951b-4ba3-4e35-b1f4-9008ae8796c5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:51.130870 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f951b-4ba3-4e35-b1f4-9008ae8796c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:51.132317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f951b-4ba3-4e35-b1f4-9008ae8796c5 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:51.138239 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 656/1291] 10.222.0.121 () {56 vars in 1121 bytes} [Thu Aug 4 10:22:50 2022] GET /identity/v3/services?name=placement => generated 127 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:51.146136 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a1d9e-7a12-46ac-b8b5-c93d37e8ff1a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:51.266057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a1d9e-7a12-46ac-b8b5-c93d37e8ff1a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:51.300875 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917046d-4527-49be-abee-09a36f723845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:51.301549 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917046d-4527-49be-abee-09a36f723845 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:51.301835 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917046d-4527-49be-abee-09a36f723845 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:51.302111 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917046d-4527-49be-abee-09a36f723845 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:51.302807 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917046d-4527-49be-abee-09a36f723845 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:51.303118 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917046d-4527-49be-abee-09a36f723845 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:51.304558 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917046d-4527-49be-abee-09a36f723845 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:51.309217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 636/1292] 10.222.0.121 () {56 vars in 1121 bytes} [Thu Aug 4 10:22:51 2022] GET /identity/v3/services?type=placement => generated 127 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:53.338892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda5d52-be0c-40d5-9e14-ac1ec55bc7a6 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:53.339566 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda5d52-be0c-40d5-9e14-ac1ec55bc7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:53.339566 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda5d52-be0c-40d5-9e14-ac1ec55bc7a6 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:53.340884 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 657/1293] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:22:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:22:53.346560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183d5af-7eae-40c3-9fa9-97604ff37f00 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:53.347053 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183d5af-7eae-40c3-9fa9-97604ff37f00 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:53.347156 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183d5af-7eae-40c3-9fa9-97604ff37f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:53.368423 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4183d5af-7eae-40c3-9fa9-97604ff37f00 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:22:53.369664 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4183d5af-7eae-40c3-9fa9-97604ff37f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:22:53.403721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:53.447367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 637/1294] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:22:53 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:22:53.471263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda5d52-be0c-40d5-9e14-ac1ec55bc7a6 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:53.508623 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:53.597696 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda5d52-be0c-40d5-9e14-ac1ec55bc7a6 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:53.640607 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c6610-23b9-458c-a171-2c8f2a90a0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:53.641827 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c6610-23b9-458c-a171-2c8f2a90a0eb None admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:53.642102 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c6610-23b9-458c-a171-2c8f2a90a0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:53.642379 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c6610-23b9-458c-a171-2c8f2a90a0eb None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:53.643118 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c6610-23b9-458c-a171-2c8f2a90a0eb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:53.644586 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c6610-23b9-458c-a171-2c8f2a90a0eb None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:53.662082 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 658/1295] 10.222.0.121 () {60 vars in 1146 bytes} [Thu Aug 4 10:22:53 2022] POST /identity/v3/services => generated 247 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:22:55.725219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a977eba6-c949-4381-ba32-2f63be3b76d0 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:55.725898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a977eba6-c949-4381-ba32-2f63be3b76d0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:55.725898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a977eba6-c949-4381-ba32-2f63be3b76d0 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:55.727195 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 638/1296] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:22:55 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:22:55.733172 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a40a3-5109-4656-9aac-f1449bc2ca4b None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:55.733733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a40a3-5109-4656-9aac-f1449bc2ca4b None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:55.733733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a40a3-5109-4656-9aac-f1449bc2ca4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:55.753958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-a30a40a3-5109-4656-9aac-f1449bc2ca4b None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:22:55.755217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-a30a40a3-5109-4656-9aac-f1449bc2ca4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:22:55.844279 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 659/1297] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:22:55 2022] POST /identity/v3/auth/tokens => generated 2226 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:22:55.869114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a977eba6-c949-4381-ba32-2f63be3b76d0 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:55.984376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a977eba6-c949-4381-ba32-2f63be3b76d0 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:56.024860 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c05aa-4d01-4feb-a051-d7aba19d5240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:56.025543 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c05aa-4d01-4feb-a051-d7aba19d5240 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:56.025831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c05aa-4d01-4feb-a051-d7aba19d5240 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:56.026109 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c05aa-4d01-4feb-a051-d7aba19d5240 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:56.026758 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677c05aa-4d01-4feb-a051-d7aba19d5240 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:56.028220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677c05aa-4d01-4feb-a051-d7aba19d5240 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:56.038511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-677c05aa-4d01-4feb-a051-d7aba19d5240 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 10:22:56.038511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:56.038511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:22:56.038511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:22:56.038511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:22:56.038511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:22:56.038511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:22:56.038511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:22:56.038511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 10:22:56.038511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:22:56.038511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:22:56.038511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:22:56.038511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:56.038511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:56.038511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:56.038511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:56.038511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:56.039907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:56.039907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:56.039907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:56.039907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:56.039907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:56.039907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:56.039907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:22:56.039907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:22:56.039907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:22:56.039907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:22:56.039907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:56.039907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:56.039907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:56.039907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:56.039907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:56.039907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:56.041217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:56.041217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:56.041217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:56.041217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:56.041217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:56.041217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:56.041217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:56.041217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:56.041217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:56.041217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:56.041217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:22:56.041217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:22:56.041217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 10:22:56.041217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:22:56.041217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 639/1298] 10.222.0.121 () {56 vars in 1122 bytes} [Thu Aug 4 10:22:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:56.046442 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a40a3-5109-4656-9aac-f1449bc2ca4b None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:56.067708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:56.180983 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a40a3-5109-4656-9aac-f1449bc2ca4b None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:56.219058 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25034829-8b67-4ec1-883c-3430a6848a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:56.219787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25034829-8b67-4ec1-883c-3430a6848a10 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:56.220174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25034829-8b67-4ec1-883c-3430a6848a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:56.220536 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25034829-8b67-4ec1-883c-3430a6848a10 None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:56.221362 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25034829-8b67-4ec1-883c-3430a6848a10 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:56.221658 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25034829-8b67-4ec1-883c-3430a6848a10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:56.223086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25034829-8b67-4ec1-883c-3430a6848a10 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:56.228747 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 660/1299] 10.222.0.121 () {56 vars in 1121 bytes} [Thu Aug 4 10:22:56 2022] GET /identity/v3/services?name=placement => generated 360 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:56.235220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c05aa-4d01-4feb-a051-d7aba19d5240 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:56.270539 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:56.345127 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c05aa-4d01-4feb-a051-d7aba19d5240 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:56.393469 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c2605-2276-47bf-afb0-5820bd4a6b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:56.394104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c2605-2276-47bf-afb0-5820bd4a6b50 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:56.394392 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c2605-2276-47bf-afb0-5820bd4a6b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:56.394670 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c2605-2276-47bf-afb0-5820bd4a6b50 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:56.395481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c2605-2276-47bf-afb0-5820bd4a6b50 None admin] RBAC: Adding query filter params (interface=public, service_id=49fe284d9a2242f2932e82b13812e64a, region_id=RegionOne) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:56.395795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c2605-2276-47bf-afb0-5820bd4a6b50 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:56.397249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c2605-2276-47bf-afb0-5820bd4a6b50 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:56.402851 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 640/1300] 10.222.0.121 () {56 vars in 1256 bytes} [Thu Aug 4 10:22:56 2022] GET /identity/v3/endpoints?service_id=49fe284d9a2242f2932e82b13812e64a&interface=public®ion_id=RegionOne => generated 195 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:56.409001 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25034829-8b67-4ec1-883c-3430a6848a10 None admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:56.497608 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:56.516774 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25034829-8b67-4ec1-883c-3430a6848a10 None admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:56.553670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27f937-be99-4064-8715-cb7f1cbac1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:56.554324 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27f937-be99-4064-8715-cb7f1cbac1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:56.554608 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27f937-be99-4064-8715-cb7f1cbac1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:56.554886 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27f937-be99-4064-8715-cb7f1cbac1e8 None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:56.555562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27f937-be99-4064-8715-cb7f1cbac1e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:56.557034 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27f937-be99-4064-8715-cb7f1cbac1e8 None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:56.564631 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 661/1301] 10.222.0.121 () {56 vars in 1092 bytes} [Thu Aug 4 10:22:56 2022] GET /identity/v3/services => generated 2201 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:22:58.626803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a3e85-050e-45b5-9fcc-151fe9635b86 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:58.627744 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a3e85-050e-45b5-9fcc-151fe9635b86 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:58.627744 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a3e85-050e-45b5-9fcc-151fe9635b86 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:58.628795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 641/1302] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:22:58 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:22:58.635492 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01483916-5ee9-4e03-a046-471376203cb6 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:58.636190 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01483916-5ee9-4e03-a046-471376203cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:58.636190 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01483916-5ee9-4e03-a046-471376203cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:58.655226 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-01483916-5ee9-4e03-a046-471376203cb6 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:22:58.656508 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-01483916-5ee9-4e03-a046-471376203cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:22:58.734567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 662/1303] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:22:58 2022] POST /identity/v3/auth/tokens => generated 2226 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 10:22:58.759806 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a3e85-050e-45b5-9fcc-151fe9635b86 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:58.863693 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:58.872153 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a3e85-050e-45b5-9fcc-151fe9635b86 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:58.908480 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4296089-8aec-40ea-bd8b-7f25aa7f521f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:58.909150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4296089-8aec-40ea-bd8b-7f25aa7f521f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:58.909433 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4296089-8aec-40ea-bd8b-7f25aa7f521f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:58.909708 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4296089-8aec-40ea-bd8b-7f25aa7f521f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:58.910361 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4296089-8aec-40ea-bd8b-7f25aa7f521f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:58.911845 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4296089-8aec-40ea-bd8b-7f25aa7f521f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:58.921657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-c4296089-8aec-40ea-bd8b-7f25aa7f521f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 10:22:58.921657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:58.921657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 10:22:58.921657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 10:22:58.921657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 10:22:58.921657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 10:22:58.921657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 10:22:58.921657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:22:58.921657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 10:22:58.921657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:22:58.921657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 10:22:58.921657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:22:58.921657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 10:22:58.921657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 10:22:58.921657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 10:22:58.921657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 10:22:58.921657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 10:22:58.922944 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 10:22:58.922944 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 10:22:58.922944 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 10:22:58.922944 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 10:22:58.922944 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 10:22:58.922944 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 10:22:58.922944 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 10:22:58.922944 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 10:22:58.922944 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 10:22:58.922944 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 10:22:58.922944 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 10:22:58.922944 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 10:22:58.922944 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 10:22:58.922944 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 10:22:58.922944 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 10:22:58.922944 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 10:22:58.924104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 10:22:58.924104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Aug 04 10:22:58.924104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 10:22:58.924104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Aug 04 10:22:58.924104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 10:22:58.924104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 10:22:58.924104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 10:22:58.924104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Aug 04 10:22:58.924104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 10:22:58.924104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Aug 04 10:22:58.924104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 10:22:58.924104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 10:22:58.924104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 10:22:58.924104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Aug 04 10:22:58.924104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 642/1304] 10.222.0.121 () {56 vars in 1122 bytes} [Thu Aug 4 10:22:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 10:22:58.929942 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01483916-5ee9-4e03-a046-471376203cb6 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:59.014618 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:22:59.039994 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01483916-5ee9-4e03-a046-471376203cb6 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:59.077217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a4873-e75a-4f22-bb87-f870472bcfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:59.077843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a4873-e75a-4f22-bb87-f870472bcfaf None admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:59.078126 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a4873-e75a-4f22-bb87-f870472bcfaf None admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:59.078394 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a4873-e75a-4f22-bb87-f870472bcfaf None admin] PATH_INFO: `/v3/services` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:59.079046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a4873-e75a-4f22-bb87-f870472bcfaf None admin] RBAC: Adding query filter params (name=placement) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:22:59.079411 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a4873-e75a-4f22-bb87-f870472bcfaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:59.080851 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a4873-e75a-4f22-bb87-f870472bcfaf None admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:59.087058 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 663/1305] 10.222.0.121 () {56 vars in 1121 bytes} [Thu Aug 4 10:22:58 2022] GET /identity/v3/services?name=placement => generated 360 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:22:59.094656 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4296089-8aec-40ea-bd8b-7f25aa7f521f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:22:59.214432 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4296089-8aec-40ea-bd8b-7f25aa7f521f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:22:59.251191 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42235d39-ab1c-410c-9981-ca52f697be40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:22:59.251819 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42235d39-ab1c-410c-9981-ca52f697be40 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:22:59.252139 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42235d39-ab1c-410c-9981-ca52f697be40 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:22:59.252401 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42235d39-ab1c-410c-9981-ca52f697be40 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:22:59.253156 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42235d39-ab1c-410c-9981-ca52f697be40 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:22:59.254695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42235d39-ab1c-410c-9981-ca52f697be40 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:22:59.277554 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 643/1306] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:22:59 2022] POST /identity/v3/endpoints => generated 333 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:23:09.083764 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=30 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:23:09.273249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:10.303345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87004e-19b4-4f0e-98e1-4913cf293dca None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:10.304047 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87004e-19b4-4f0e-98e1-4913cf293dca None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:10.304047 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87004e-19b4-4f0e-98e1-4913cf293dca None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:10.308387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 664/1307] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:24:10 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:24:10.310759 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e918b1-bcf6-43be-9618-c4cb9bf1ec5a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:10.311273 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e918b1-bcf6-43be-9618-c4cb9bf1ec5a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:10.311375 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e918b1-bcf6-43be-9618-c4cb9bf1ec5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:10.417923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f3e918b1-bcf6-43be-9618-c4cb9bf1ec5a None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:10.419274 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f3e918b1-bcf6-43be-9618-c4cb9bf1ec5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:10.610181 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 644/1308] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:24:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:10.681875 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8c7a8-2747-4563-b2f8-2ff67907bb0f None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:10.681875 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8c7a8-2747-4563-b2f8-2ff67907bb0f None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:10.681875 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8c7a8-2747-4563-b2f8-2ff67907bb0f None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:10.681875 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 665/1309] 10.222.0.121 () {54 vars in 970 bytes} [Thu Aug 4 10:24:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:24:10.696929 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6cd3c-eae0-4bc3-a73f-4e0f92c5cf67 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:10.698146 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6cd3c-eae0-4bc3-a73f-4e0f92c5cf67 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:10.701892 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6cd3c-eae0-4bc3-a73f-4e0f92c5cf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:10.842444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-64a6cd3c-eae0-4bc3-a73f-4e0f92c5cf67 None None] MFA Rules not processed for user `90c7eaf4500f4ef496ba72405522006d`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:10.844081 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-64a6cd3c-eae0-4bc3-a73f-4e0f92c5cf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:10.871203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:10.942239 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 645/1310] 10.222.0.121 () {58 vars in 1067 bytes} [Thu Aug 4 10:24:10 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:10.951938 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8c7a8-2747-4563-b2f8-2ff67907bb0f None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:10.953925 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6ff8c7a8-2747-4563-b2f8-2ff67907bb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:11.072231 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8c7a8-2747-4563-b2f8-2ff67907bb0f None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:11.170939 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2110eacc-e7b7-460d-88fb-a334c5f058fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:11.171747 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110eacc-e7b7-460d-88fb-a334c5f058fd service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:11.172270 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110eacc-e7b7-460d-88fb-a334c5f058fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:11.172639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110eacc-e7b7-460d-88fb-a334c5f058fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:11.174564 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-2110eacc-e7b7-460d-88fb-a334c5f058fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:11.260692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:11.328418 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2110eacc-e7b7-460d-88fb-a334c5f058fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:11.330693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2110eacc-e7b7-460d-88fb-a334c5f058fd service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:11.353933 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 666/1311] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:24:10 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:24:13.794814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32ed3d-3a56-4146-b8bb-a21e8c3d0bbf None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:13.795409 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32ed3d-3a56-4146-b8bb-a21e8c3d0bbf None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:13.795409 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32ed3d-3a56-4146-b8bb-a21e8c3d0bbf None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:13.796654 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 646/1312] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:24:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:24:13.802720 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61686941-71c9-4e29-bb2f-456fecfaf10c None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:13.803186 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61686941-71c9-4e29-bb2f-456fecfaf10c None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:13.803186 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61686941-71c9-4e29-bb2f-456fecfaf10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:13.889670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-61686941-71c9-4e29-bb2f-456fecfaf10c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:13.890932 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-61686941-71c9-4e29-bb2f-456fecfaf10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:14.015835 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 667/1313] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:24:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:14.068536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e8fb8-29c4-4e92-a43d-488771766d10 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:14.068810 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e8fb8-29c4-4e92-a43d-488771766d10 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:14.069082 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e8fb8-29c4-4e92-a43d-488771766d10 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:14.070539 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 647/1314] 10.222.0.121 () {54 vars in 970 bytes} [Thu Aug 4 10:24:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 10:24:14.078930 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876ad9e-31f1-48d2-bd23-38ede5405d44 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:14.079214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876ad9e-31f1-48d2-bd23-38ede5405d44 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:14.079489 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876ad9e-31f1-48d2-bd23-38ede5405d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:14.092271 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:14.171156 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-8876ad9e-31f1-48d2-bd23-38ede5405d44 None None] MFA Rules not processed for user `90c7eaf4500f4ef496ba72405522006d`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:14.172846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-8876ad9e-31f1-48d2-bd23-38ede5405d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:14.285649 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 668/1315] 10.222.0.121 () {58 vars in 1067 bytes} [Thu Aug 4 10:24:14 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:14.296056 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e8fb8-29c4-4e92-a43d-488771766d10 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:14.298234 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-297e8fb8-29c4-4e92-a43d-488771766d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:14.422806 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e8fb8-29c4-4e92-a43d-488771766d10 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:14.513951 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:14.523186 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15545692-9ed8-43d7-b79b-2e247ec8ac06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:14.523837 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15545692-9ed8-43d7-b79b-2e247ec8ac06 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:14.524247 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15545692-9ed8-43d7-b79b-2e247ec8ac06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:14.524486 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15545692-9ed8-43d7-b79b-2e247ec8ac06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:14.526187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-15545692-9ed8-43d7-b79b-2e247ec8ac06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:14.668801 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15545692-9ed8-43d7-b79b-2e247ec8ac06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:14.670331 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15545692-9ed8-43d7-b79b-2e247ec8ac06 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:14.691419 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 648/1316] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:24:14 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:24:17.310088 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb718d-a7f6-4d4a-bdfb-8ad9e1ff8a37 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:17.310778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb718d-a7f6-4d4a-bdfb-8ad9e1ff8a37 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:17.310778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb718d-a7f6-4d4a-bdfb-8ad9e1ff8a37 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:17.312030 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 669/1317] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:24:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:24:17.318126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e6d5c-8daf-43fb-80c0-4a4a871306b7 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:17.318682 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e6d5c-8daf-43fb-80c0-4a4a871306b7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:17.318682 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e6d5c-8daf-43fb-80c0-4a4a871306b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:17.393133 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-917e6d5c-8daf-43fb-80c0-4a4a871306b7 None None] MFA Rules not processed for user `5b20a8c5e8314c8794af566d108815a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:17.394407 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-917e6d5c-8daf-43fb-80c0-4a4a871306b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:17.448635 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:17.497829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 649/1318] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:24:17 2022] POST /identity/v3/auth/tokens => generated 3253 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:17.539104 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb718d-a7f6-4d4a-bdfb-8ad9e1ff8a37 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:17.540753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-4aeb718d-a7f6-4d4a-bdfb-8ad9e1ff8a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:17.654900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb718d-a7f6-4d4a-bdfb-8ad9e1ff8a37 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:17.657980 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:17.742843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc69538-938b-4481-a6c3-531f8647d08d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:17.743472 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc69538-938b-4481-a6c3-531f8647d08d service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:17.743750 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc69538-938b-4481-a6c3-531f8647d08d service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:17.744057 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc69538-938b-4481-a6c3-531f8647d08d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:17.745710 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-abc69538-938b-4481-a6c3-531f8647d08d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:17.866890 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc69538-938b-4481-a6c3-531f8647d08d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:17.868409 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc69538-938b-4481-a6c3-531f8647d08d service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:17.888512 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 670/1319] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:24:17 2022] GET /identity/v3/auth/tokens => generated 3253 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:24:20.551991 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e50bb2-a3e6-4c3e-a47d-2acc48c2ae8e None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:20.552635 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e50bb2-a3e6-4c3e-a47d-2acc48c2ae8e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:20.552635 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e50bb2-a3e6-4c3e-a47d-2acc48c2ae8e None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:20.553933 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 650/1320] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:24:20 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:24:20.560009 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca2919-3d14-4438-9b45-a44a917fef09 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:20.560567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca2919-3d14-4438-9b45-a44a917fef09 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:20.560567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca2919-3d14-4438-9b45-a44a917fef09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:20.610408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:20.639988 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-d0ca2919-3d14-4438-9b45-a44a917fef09 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:20.641260 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d0ca2919-3d14-4438-9b45-a44a917fef09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:20.744591 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 671/1321] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:24:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:20.796068 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e50bb2-a3e6-4c3e-a47d-2acc48c2ae8e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:20.797710 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-81e50bb2-a3e6-4c3e-a47d-2acc48c2ae8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:20.920627 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e50bb2-a3e6-4c3e-a47d-2acc48c2ae8e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:21.013386 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ee958-b8c0-44cb-a33a-c0538062f972 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:21.014028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ee958-b8c0-44cb-a33a-c0538062f972 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:21.014305 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ee958-b8c0-44cb-a33a-c0538062f972 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:21.014574 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ee958-b8c0-44cb-a33a-c0538062f972 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:21.016642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-812ee958-b8c0-44cb-a33a-c0538062f972 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:21.025932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:21.158392 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ee958-b8c0-44cb-a33a-c0538062f972 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:21.159866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ee958-b8c0-44cb-a33a-c0538062f972 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:21.180056 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 651/1322] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:24:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:24:23.913734 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363172a1-8b98-4c96-a77e-6e13c1cb0161 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:23.914391 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363172a1-8b98-4c96-a77e-6e13c1cb0161 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:23.914391 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363172a1-8b98-4c96-a77e-6e13c1cb0161 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:23.915795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 672/1323] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:24:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:24:23.921355 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee40a9-e61d-40cc-b837-2a530b8cd39c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:23.921876 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee40a9-e61d-40cc-b837-2a530b8cd39c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:23.921999 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee40a9-e61d-40cc-b837-2a530b8cd39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:23.995890 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c7ee40a9-e61d-40cc-b837-2a530b8cd39c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:23.997291 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c7ee40a9-e61d-40cc-b837-2a530b8cd39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:24.069406 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:24.124432 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 652/1324] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:24:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:24.169602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363172a1-8b98-4c96-a77e-6e13c1cb0161 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:24.172011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-363172a1-8b98-4c96-a77e-6e13c1cb0161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:24.292196 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363172a1-8b98-4c96-a77e-6e13c1cb0161 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:24.305608 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:24.373685 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf636d-b709-468c-8fb1-3d9fa6789a6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:24.374325 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf636d-b709-468c-8fb1-3d9fa6789a6c service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:24.374610 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf636d-b709-468c-8fb1-3d9fa6789a6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:24.374897 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf636d-b709-468c-8fb1-3d9fa6789a6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:24.376637 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-0ebf636d-b709-468c-8fb1-3d9fa6789a6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:24.519737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebf636d-b709-468c-8fb1-3d9fa6789a6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:24.521372 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebf636d-b709-468c-8fb1-3d9fa6789a6c service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:24.542842 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 673/1325] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:24:24 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:24:26.613896 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90283502-2188-4502-bad4-eaab4799b149 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:26.614571 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90283502-2188-4502-bad4-eaab4799b149 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:26.614571 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90283502-2188-4502-bad4-eaab4799b149 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:26.615805 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 653/1326] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:24:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:24:26.621242 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e696f-e9bc-4d9e-8b11-af929ce0cea1 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:26.621718 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e696f-e9bc-4d9e-8b11-af929ce0cea1 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:26.621836 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e696f-e9bc-4d9e-8b11-af929ce0cea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:26.673267 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:26.692353 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-037e696f-e9bc-4d9e-8b11-af929ce0cea1 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:26.693604 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-037e696f-e9bc-4d9e-8b11-af929ce0cea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:26.797549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 674/1327] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:24:26 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:26.834128 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90283502-2188-4502-bad4-eaab4799b149 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:26.835763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-90283502-2188-4502-bad4-eaab4799b149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:26.953025 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90283502-2188-4502-bad4-eaab4799b149 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:27.041860 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af9937-a591-47e1-8553-be9523d6c588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:27.042500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af9937-a591-47e1-8553-be9523d6c588 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:27.042790 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af9937-a591-47e1-8553-be9523d6c588 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:27.043073 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af9937-a591-47e1-8553-be9523d6c588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:27.044821 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-78af9937-a591-47e1-8553-be9523d6c588 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:27.063888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:27.166819 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af9937-a591-47e1-8553-be9523d6c588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:27.168237 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af9937-a591-47e1-8553-be9523d6c588 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:27.194487 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 654/1328] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:24:26 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:24:29.321988 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55881254-84f7-4722-b297-be7c3168924b None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:29.322664 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55881254-84f7-4722-b297-be7c3168924b None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:29.322664 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55881254-84f7-4722-b297-be7c3168924b None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:29.323908 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 675/1329] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:24:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:24:29.330275 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137066a-fa10-4945-9a7f-7d820d655e6d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:29.330927 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137066a-fa10-4945-9a7f-7d820d655e6d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:29.330927 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137066a-fa10-4945-9a7f-7d820d655e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:29.405555 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f137066a-fa10-4945-9a7f-7d820d655e6d None None] MFA Rules not processed for user `5b20a8c5e8314c8794af566d108815a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:29.406823 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f137066a-fa10-4945-9a7f-7d820d655e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:29.480114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:29.513134 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 655/1330] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:24:29 2022] POST /identity/v3/auth/tokens => generated 3253 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:24:29.551814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55881254-84f7-4722-b297-be7c3168924b None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:29.553441 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-55881254-84f7-4722-b297-be7c3168924b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:29.669207 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55881254-84f7-4722-b297-be7c3168924b None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:29.691680 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:29.750169 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1015e-cfae-4fcc-ad01-b66e45aab3cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:29.750811 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1015e-cfae-4fcc-ad01-b66e45aab3cb service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:29.751093 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1015e-cfae-4fcc-ad01-b66e45aab3cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:29.751338 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1015e-cfae-4fcc-ad01-b66e45aab3cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:29.753049 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-4ef1015e-cfae-4fcc-ad01-b66e45aab3cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:29.887265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1015e-cfae-4fcc-ad01-b66e45aab3cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:29.888928 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1015e-cfae-4fcc-ad01-b66e45aab3cb service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:29.910526 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 676/1331] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:24:29 2022] GET /identity/v3/auth/tokens => generated 3253 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:24:32.217291 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f698d-7acd-42a1-9ada-12f144b18630 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:32.217921 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f698d-7acd-42a1-9ada-12f144b18630 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:32.217921 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f698d-7acd-42a1-9ada-12f144b18630 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:32.219202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 656/1332] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:24:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:24:32.228817 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7884a-5403-4d3f-a592-ef2586115a70 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:32.229573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7884a-5403-4d3f-a592-ef2586115a70 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:32.229729 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7884a-5403-4d3f-a592-ef2586115a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:32.306146 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:32.308945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-93b7884a-5403-4d3f-a592-ef2586115a70 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:32.311079 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-93b7884a-5403-4d3f-a592-ef2586115a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:32.433678 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 677/1333] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:24:32 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:32.470952 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f698d-7acd-42a1-9ada-12f144b18630 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:32.472521 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6b3f698d-7acd-42a1-9ada-12f144b18630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:32.590048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f698d-7acd-42a1-9ada-12f144b18630 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:32.667033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8419908-1668-41a9-b5c6-cd5aadf9781e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:32.667669 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8419908-1668-41a9-b5c6-cd5aadf9781e service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:32.667986 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8419908-1668-41a9-b5c6-cd5aadf9781e service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:32.668259 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8419908-1668-41a9-b5c6-cd5aadf9781e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:32.669919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f8419908-1668-41a9-b5c6-cd5aadf9781e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:32.700409 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:32.800814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8419908-1668-41a9-b5c6-cd5aadf9781e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:32.802150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8419908-1668-41a9-b5c6-cd5aadf9781e service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:32.820918 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 657/1334] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:24:32 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:24:34.931374 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdc0b6-443d-4969-978a-bd52937664c4 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:34.932001 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdc0b6-443d-4969-978a-bd52937664c4 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:34.932001 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdc0b6-443d-4969-978a-bd52937664c4 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:34.933927 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 678/1335] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:24:34 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:24:34.942535 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55ecba-c799-444f-a327-9799dacc0eac None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:34.942996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55ecba-c799-444f-a327-9799dacc0eac None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:34.943101 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55ecba-c799-444f-a327-9799dacc0eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:35.022117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-eb55ecba-c799-444f-a327-9799dacc0eac None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:35.023363 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-eb55ecba-c799-444f-a327-9799dacc0eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:35.115642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:35.139570 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 658/1336] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:24:34 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:35.195876 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdc0b6-443d-4969-978a-bd52937664c4 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:35.198347 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-62cdc0b6-443d-4969-978a-bd52937664c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:35.324172 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdc0b6-443d-4969-978a-bd52937664c4 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:35.365429 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:35.416739 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc3b757-a2a7-4804-b596-b8f5fbe598a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:35.417359 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3b757-a2a7-4804-b596-b8f5fbe598a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:35.417653 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3b757-a2a7-4804-b596-b8f5fbe598a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:35.417961 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3b757-a2a7-4804-b596-b8f5fbe598a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:35.419591 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-9cc3b757-a2a7-4804-b596-b8f5fbe598a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:35.535706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc3b757-a2a7-4804-b596-b8f5fbe598a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:35.537106 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc3b757-a2a7-4804-b596-b8f5fbe598a5 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:35.555689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 679/1337] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:24:35 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:24:37.979098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a292def-b47a-428d-b612-ffbf3ba1be88 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:37.980061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a292def-b47a-428d-b612-ffbf3ba1be88 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:37.980061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a292def-b47a-428d-b612-ffbf3ba1be88 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:37.982224 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 659/1338] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:24:37 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:24:37.991725 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28e3ac-180e-4bf4-a764-f7664f5f1dbb None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:37.992535 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28e3ac-180e-4bf4-a764-f7664f5f1dbb None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:37.992721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28e3ac-180e-4bf4-a764-f7664f5f1dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:38.075074 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-8f28e3ac-180e-4bf4-a764-f7664f5f1dbb None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:38.076472 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-8f28e3ac-180e-4bf4-a764-f7664f5f1dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:38.085338 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:38.186729 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 680/1339] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:24:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:38.226455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a292def-b47a-428d-b612-ffbf3ba1be88 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:38.228089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5a292def-b47a-428d-b612-ffbf3ba1be88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:38.354002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a292def-b47a-428d-b612-ffbf3ba1be88 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:38.429605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09adb7-e4ad-4167-a988-e53383914cae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:38.430262 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09adb7-e4ad-4167-a988-e53383914cae service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:38.430490 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09adb7-e4ad-4167-a988-e53383914cae service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:38.430797 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09adb7-e4ad-4167-a988-e53383914cae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:38.432488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cd09adb7-e4ad-4167-a988-e53383914cae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:38.468943 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:38.550657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd09adb7-e4ad-4167-a988-e53383914cae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:38.552221 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd09adb7-e4ad-4167-a988-e53383914cae service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:38.571265 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 660/1340] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:24:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:24:42.724356 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1fe44-2e09-4565-aa48-b3b1ebcfd1ee None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:42.724983 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1fe44-2e09-4565-aa48-b3b1ebcfd1ee None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:42.724983 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1fe44-2e09-4565-aa48-b3b1ebcfd1ee None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:42.726315 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 681/1341] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:24:42 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:24:42.732205 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1830462-5aa4-493f-8399-70ab03dbbfb7 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:42.732682 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1830462-5aa4-493f-8399-70ab03dbbfb7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:42.732780 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1830462-5aa4-493f-8399-70ab03dbbfb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:42.806676 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b1830462-5aa4-493f-8399-70ab03dbbfb7 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:42.807938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b1830462-5aa4-493f-8399-70ab03dbbfb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:42.908600 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:42.923736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 661/1342] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:24:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:42.969300 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1fe44-2e09-4565-aa48-b3b1ebcfd1ee None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:42.970980 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-87e1fe44-2e09-4565-aa48-b3b1ebcfd1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:43.088130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1fe44-2e09-4565-aa48-b3b1ebcfd1ee None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:43.131144 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:43.170765 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c347df-2854-492b-82c0-765ce21105b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:43.171515 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c347df-2854-492b-82c0-765ce21105b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:43.171991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c347df-2854-492b-82c0-765ce21105b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:43.172506 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c347df-2854-492b-82c0-765ce21105b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:43.174823 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-24c347df-2854-492b-82c0-765ce21105b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:43.298454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c347df-2854-492b-82c0-765ce21105b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:43.299816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c347df-2854-492b-82c0-765ce21105b6 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:43.322300 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 682/1343] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:24:42 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:24:46.161457 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185935d-1a7b-49bc-b9b8-5b04f92dee86 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:46.162170 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185935d-1a7b-49bc-b9b8-5b04f92dee86 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:46.162170 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185935d-1a7b-49bc-b9b8-5b04f92dee86 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:46.163492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 662/1344] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:24:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:24:46.170696 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7227d91-f3e6-4f18-949b-fb958f87ed47 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:46.171353 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7227d91-f3e6-4f18-949b-fb958f87ed47 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:46.171353 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7227d91-f3e6-4f18-949b-fb958f87ed47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:46.248072 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-d7227d91-f3e6-4f18-949b-fb958f87ed47 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:46.249340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d7227d91-f3e6-4f18-949b-fb958f87ed47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:46.269663 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:46.356571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 683/1345] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:24:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:46.396357 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185935d-1a7b-49bc-b9b8-5b04f92dee86 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:46.398478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d185935d-1a7b-49bc-b9b8-5b04f92dee86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:46.514302 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185935d-1a7b-49bc-b9b8-5b04f92dee86 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:46.592600 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2a341-3f2d-4fb6-b885-be70ec532787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:46.593365 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2a341-3f2d-4fb6-b885-be70ec532787 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:46.593586 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2a341-3f2d-4fb6-b885-be70ec532787 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:46.593860 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2a341-3f2d-4fb6-b885-be70ec532787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:46.595570 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-15b2a341-3f2d-4fb6-b885-be70ec532787 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:46.641865 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:46.718424 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b2a341-3f2d-4fb6-b885-be70ec532787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:46.719965 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b2a341-3f2d-4fb6-b885-be70ec532787 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:46.745550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 663/1346] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:24:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:24:51.050886 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801cb60-e4eb-49a3-9a0d-0131df085edb None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:51.051539 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801cb60-e4eb-49a3-9a0d-0131df085edb None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:51.051539 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801cb60-e4eb-49a3-9a0d-0131df085edb None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:51.052920 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 684/1347] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:24:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:24:51.058646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a214606-75df-44b4-9856-15d69d0b420b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:51.059181 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a214606-75df-44b4-9856-15d69d0b420b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:51.059284 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a214606-75df-44b4-9856-15d69d0b420b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:51.140301 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9a214606-75df-44b4-9856-15d69d0b420b None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:51.141703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9a214606-75df-44b4-9856-15d69d0b420b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:51.272450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:51.275816 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 664/1348] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:24:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:24:51.316210 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801cb60-e4eb-49a3-9a0d-0131df085edb None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:24:51.317911 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-2801cb60-e4eb-49a3-9a0d-0131df085edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:51.439221 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801cb60-e4eb-49a3-9a0d-0131df085edb None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:24:51.491470 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:51.534154 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e64a5f-24bc-45b2-8197-a43266fbe1a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:24:51.534801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e64a5f-24bc-45b2-8197-a43266fbe1a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:51.535218 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e64a5f-24bc-45b2-8197-a43266fbe1a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:51.535419 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e64a5f-24bc-45b2-8197-a43266fbe1a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:51.538064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-a4e64a5f-24bc-45b2-8197-a43266fbe1a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:51.680683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e64a5f-24bc-45b2-8197-a43266fbe1a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:24:51.682175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e64a5f-24bc-45b2-8197-a43266fbe1a3 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:24:51.702854 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 685/1349] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:24:51 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:24:57.017949 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e7b0c-dbfa-46f2-b541-d50d3fefd865 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:57.018445 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e7b0c-dbfa-46f2-b541-d50d3fefd865 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:57.018531 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e7b0c-dbfa-46f2-b541-d50d3fefd865 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:57.020021 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 665/1350] 10.222.0.121 () {54 vars in 926 bytes} [Thu Aug 4 10:24:57 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:24:57.026421 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfdb4a-52d1-4872-9301-6bd45e844448 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:24:57.026955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfdb4a-52d1-4872-9301-6bd45e844448 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:24:57.026955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfdb4a-52d1-4872-9301-6bd45e844448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:24:57.135462 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-88cfdb4a-52d1-4872-9301-6bd45e844448 None None] MFA Rules not processed for user `01c7cfbe911c4378857848966f4f8d98`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:24:57.136505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-88cfdb4a-52d1-4872-9301-6bd45e844448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:24:57.159319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:24:57.294631 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 686/1351] 10.222.0.121 () {58 vars in 1023 bytes} [Thu Aug 4 10:24:57 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:02.003969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0d09d-7b85-4ff6-94d2-40c1f20e9692 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:02.004662 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0d09d-7b85-4ff6-94d2-40c1f20e9692 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:02.004662 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0d09d-7b85-4ff6-94d2-40c1f20e9692 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:02.006349 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 666/1352] 10.222.0.121 () {54 vars in 926 bytes} [Thu Aug 4 10:25:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 04 10:25:02.011599 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751838eb-be5f-4fa9-994e-1d2a677ae917 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:02.012218 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751838eb-be5f-4fa9-994e-1d2a677ae917 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:02.012218 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751838eb-be5f-4fa9-994e-1d2a677ae917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:02.101941 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-751838eb-be5f-4fa9-994e-1d2a677ae917 None None] MFA Rules not processed for user `01c7cfbe911c4378857848966f4f8d98`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:02.103291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-751838eb-be5f-4fa9-994e-1d2a677ae917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:02.185333 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:02.229005 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 687/1353] 10.222.0.121 () {58 vars in 1023 bytes} [Thu Aug 4 10:25:02 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:03.580676 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f192f-6503-406d-8e22-5f6409aafb94 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:03.581290 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f192f-6503-406d-8e22-5f6409aafb94 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:03.581290 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f192f-6503-406d-8e22-5f6409aafb94 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:03.582939 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 667/1354] 10.222.0.121 () {54 vars in 926 bytes} [Thu Aug 4 10:25:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:25:03.588504 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f231d8-a7fe-4c7c-8b46-a09c196657d9 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:03.588996 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f231d8-a7fe-4c7c-8b46-a09c196657d9 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:03.589081 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f231d8-a7fe-4c7c-8b46-a09c196657d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:03.670539 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-53f231d8-a7fe-4c7c-8b46-a09c196657d9 None None] MFA Rules not processed for user `01c7cfbe911c4378857848966f4f8d98`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:03.671784 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-53f231d8-a7fe-4c7c-8b46-a09c196657d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:03.788946 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 688/1355] 10.222.0.121 () {58 vars in 1023 bytes} [Thu Aug 4 10:25:03 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:07.253965 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053af8e-1a9a-4def-b8db-fa93289f337c None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:07.254612 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053af8e-1a9a-4def-b8db-fa93289f337c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:07.254612 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053af8e-1a9a-4def-b8db-fa93289f337c None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:07.255967 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 668/1356] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:25:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:25:07.261761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f68314-54bb-4508-b436-5ae129061094 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:07.262336 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f68314-54bb-4508-b436-5ae129061094 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:07.262336 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f68314-54bb-4508-b436-5ae129061094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:07.310491 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:07.345210 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-b9f68314-54bb-4508-b436-5ae129061094 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:07.346529 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-b9f68314-54bb-4508-b436-5ae129061094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:07.480832 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 689/1357] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:25:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:07.567077 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855c681-b361-4e48-a08b-e6ebe6878514 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:07.568359 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855c681-b361-4e48-a08b-e6ebe6878514 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:07.568359 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855c681-b361-4e48-a08b-e6ebe6878514 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:07.569687 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 669/1358] 10.222.0.121 () {54 vars in 967 bytes} [Thu Aug 4 10:25:07 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:25:07.588004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38cf5a-fc49-492b-affc-c46ada8402ad None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:07.588004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38cf5a-fc49-492b-affc-c46ada8402ad None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:07.588004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38cf5a-fc49-492b-affc-c46ada8402ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:07.679923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-8d38cf5a-fc49-492b-affc-c46ada8402ad None None] MFA Rules not processed for user `0d947aeb749546309a8d4b7938087ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:07.681312 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-8d38cf5a-fc49-492b-affc-c46ada8402ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:07.735381 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:07.825027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 690/1359] 10.222.0.121 () {58 vars in 1064 bytes} [Thu Aug 4 10:25:07 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:07.833637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855c681-b361-4e48-a08b-e6ebe6878514 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:07.835395 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d855c681-b361-4e48-a08b-e6ebe6878514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:08.003544 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855c681-b361-4e48-a08b-e6ebe6878514 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:08.082848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:08.128679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ed417-223b-41b8-ab97-4c63a3ea7601 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:08.129508 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ed417-223b-41b8-ab97-4c63a3ea7601 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:08.129815 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ed417-223b-41b8-ab97-4c63a3ea7601 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:08.130086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ed417-223b-41b8-ab97-4c63a3ea7601 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:08.131863 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ea7ed417-223b-41b8-ab97-4c63a3ea7601 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:08.226631 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ed417-223b-41b8-ab97-4c63a3ea7601 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:08.228352 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ed417-223b-41b8-ab97-4c63a3ea7601 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:08.247992 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 670/1360] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:25:07 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:10.523409 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3b5d0-3811-486f-9cb0-62d02295ef69 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:10.524008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3b5d0-3811-486f-9cb0-62d02295ef69 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:10.524008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3b5d0-3811-486f-9cb0-62d02295ef69 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:10.525635 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 691/1361] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:25:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:25:10.530811 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4744ce9-0f03-4461-89b9-34a1fd3f6312 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:10.531273 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4744ce9-0f03-4461-89b9-34a1fd3f6312 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:10.531374 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4744ce9-0f03-4461-89b9-34a1fd3f6312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:10.612610 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d4744ce9-0f03-4461-89b9-34a1fd3f6312 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:10.614296 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d4744ce9-0f03-4461-89b9-34a1fd3f6312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:10.655801 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:10.733126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 671/1362] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:25:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:10.780514 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3b5d0-3811-486f-9cb0-62d02295ef69 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:10.782566 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-44f3b5d0-3811-486f-9cb0-62d02295ef69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:10.961213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3b5d0-3811-486f-9cb0-62d02295ef69 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:10.973547 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:11.102448 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac072130-8893-4bf7-946a-b19f39cbf3a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:11.103543 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac072130-8893-4bf7-946a-b19f39cbf3a2 service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:11.104057 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac072130-8893-4bf7-946a-b19f39cbf3a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:11.104618 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac072130-8893-4bf7-946a-b19f39cbf3a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:11.106610 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-ac072130-8893-4bf7-946a-b19f39cbf3a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:11.238637 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac072130-8893-4bf7-946a-b19f39cbf3a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:11.240354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac072130-8893-4bf7-946a-b19f39cbf3a2 service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:11.265821 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 692/1363] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:25:10 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:13.650471 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbcaa2-f02a-4a4d-a965-b2af38f72bc9 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:13.651098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbcaa2-f02a-4a4d-a965-b2af38f72bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:13.651098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbcaa2-f02a-4a4d-a965-b2af38f72bc9 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:13.657040 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 672/1364] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:25:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:25:13.658308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa2e58-b29f-44a7-a355-dbc28818fe27 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:13.658758 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa2e58-b29f-44a7-a355-dbc28818fe27 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:13.658849 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa2e58-b29f-44a7-a355-dbc28818fe27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:13.676858 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:13.744547 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-6aaa2e58-b29f-44a7-a355-dbc28818fe27 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:13.746115 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6aaa2e58-b29f-44a7-a355-dbc28818fe27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:13.864070 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 693/1365] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:25:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:13.913771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706c334-8f53-4923-bdb2-df9099b9f6b8 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:13.914126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706c334-8f53-4923-bdb2-df9099b9f6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:13.914410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706c334-8f53-4923-bdb2-df9099b9f6b8 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:13.915848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 673/1366] 10.222.0.121 () {54 vars in 967 bytes} [Thu Aug 4 10:25:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:25:13.924370 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36a4f8-9e84-4c7d-a6c4-c0a2313621b8 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:13.925842 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36a4f8-9e84-4c7d-a6c4-c0a2313621b8 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:13.926351 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36a4f8-9e84-4c7d-a6c4-c0a2313621b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:14.026983 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-1a36a4f8-9e84-4c7d-a6c4-c0a2313621b8 None None] MFA Rules not processed for user `0d947aeb749546309a8d4b7938087ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:14.027497 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-1a36a4f8-9e84-4c7d-a6c4-c0a2313621b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:14.068340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:14.174173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 694/1367] 10.222.0.121 () {58 vars in 1064 bytes} [Thu Aug 4 10:25:13 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:14.182409 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706c334-8f53-4923-bdb2-df9099b9f6b8 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:14.184024 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d706c334-8f53-4923-bdb2-df9099b9f6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:14.351425 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706c334-8f53-4923-bdb2-df9099b9f6b8 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:14.375798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:14.483592 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e38955-6fd1-46b6-abd4-a363ca54d5fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:14.484136 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e38955-6fd1-46b6-abd4-a363ca54d5fe service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:14.484424 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e38955-6fd1-46b6-abd4-a363ca54d5fe service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:14.484712 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e38955-6fd1-46b6-abd4-a363ca54d5fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:14.486728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d5e38955-6fd1-46b6-abd4-a363ca54d5fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:14.593715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e38955-6fd1-46b6-abd4-a363ca54d5fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:14.595625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e38955-6fd1-46b6-abd4-a363ca54d5fe service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:14.617375 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 674/1368] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:25:14 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:15.873762 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1820612-f920-4ceb-bedd-10d4c708bd35 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:15.874348 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1820612-f920-4ceb-bedd-10d4c708bd35 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:15.874348 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1820612-f920-4ceb-bedd-10d4c708bd35 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:15.876520 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 695/1369] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:25:15 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:25:15.882535 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6fbcc3-6da3-4875-9ca1-88964fefbe47 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:15.883466 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6fbcc3-6da3-4875-9ca1-88964fefbe47 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:15.884127 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6fbcc3-6da3-4875-9ca1-88964fefbe47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:15.922079 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:15.968333 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-cb6fbcc3-6da3-4875-9ca1-88964fefbe47 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:15.969571 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cb6fbcc3-6da3-4875-9ca1-88964fefbe47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:16.098473 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 675/1370] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:25:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:16.448808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe74e27-bb73-46a7-928c-b3917eb93b6e None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:16.448808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe74e27-bb73-46a7-928c-b3917eb93b6e None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:16.448808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe74e27-bb73-46a7-928c-b3917eb93b6e None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:16.452134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 696/1371] 10.222.0.121 () {54 vars in 969 bytes} [Thu Aug 4 10:25:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:25:16.455534 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e68bd-834d-49ea-8d78-38ddef474baf None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:16.455811 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e68bd-834d-49ea-8d78-38ddef474baf None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:16.456123 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e68bd-834d-49ea-8d78-38ddef474baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:16.544029 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-917e68bd-834d-49ea-8d78-38ddef474baf None None] MFA Rules not processed for user `c19bb76d0e2544299916007fbfa62578`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:16.545371 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-917e68bd-834d-49ea-8d78-38ddef474baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:16.613059 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:16.649097 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 676/1372] 10.222.0.121 () {58 vars in 1066 bytes} [Thu Aug 4 10:25:16 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:16.657820 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe74e27-bb73-46a7-928c-b3917eb93b6e None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:16.659514 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-ebe74e27-bb73-46a7-928c-b3917eb93b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:16.729731 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e86cd-531e-4601-ac4c-7ca12d6fa7b1 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:16.729731 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e86cd-531e-4601-ac4c-7ca12d6fa7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:16.730043 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e86cd-531e-4601-ac4c-7ca12d6fa7b1 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:16.731580 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 677/1373] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:25:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:25:16.737874 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d60ab-baee-4055-98fe-d8920d66c62f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:16.738273 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d60ab-baee-4055-98fe-d8920d66c62f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:16.738531 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d60ab-baee-4055-98fe-d8920d66c62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:16.792878 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe74e27-bb73-46a7-928c-b3917eb93b6e None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:16.822482 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-049d60ab-baee-4055-98fe-d8920d66c62f None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:16.823866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-049d60ab-baee-4055-98fe-d8920d66c62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:16.840353 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:16.881116 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263b371-3ca6-4bd8-a361-400cbedfb048 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:16.881820 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263b371-3ca6-4bd8-a361-400cbedfb048 service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:16.882037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263b371-3ca6-4bd8-a361-400cbedfb048 service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:16.882315 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263b371-3ca6-4bd8-a361-400cbedfb048 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:16.884149 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-1263b371-3ca6-4bd8-a361-400cbedfb048 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:16.950176 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 678/1374] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:25:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:16.997501 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d60ab-baee-4055-98fe-d8920d66c62f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:16.998742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-049d60ab-baee-4055-98fe-d8920d66c62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:17.016691 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1263b371-3ca6-4bd8-a361-400cbedfb048 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:17.018945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1263b371-3ca6-4bd8-a361-400cbedfb048 service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:17.041438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 697/1375] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:25:16 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:17.068788 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:17.175150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d60ab-baee-4055-98fe-d8920d66c62f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:17.337683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b2cb6-cfe9-41ba-a85d-0aba19cc4caf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:17.338316 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b2cb6-cfe9-41ba-a85d-0aba19cc4caf service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:17.338602 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b2cb6-cfe9-41ba-a85d-0aba19cc4caf service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:17.338878 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b2cb6-cfe9-41ba-a85d-0aba19cc4caf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:17.340678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d03b2cb6-cfe9-41ba-a85d-0aba19cc4caf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:17.391829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:17.437303 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b2cb6-cfe9-41ba-a85d-0aba19cc4caf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:17.438950 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b2cb6-cfe9-41ba-a85d-0aba19cc4caf service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:17.457237 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 679/1376] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:25:16 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:25:19.543896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe8a0e-3291-4960-9d7e-f78a9ab2c391 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:19.544477 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe8a0e-3291-4960-9d7e-f78a9ab2c391 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:19.544575 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe8a0e-3291-4960-9d7e-f78a9ab2c391 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:19.545933 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 698/1377] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:25:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:25:19.552040 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74e007-a1c3-4936-ab81-dff09b961ad0 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:19.552462 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74e007-a1c3-4936-ab81-dff09b961ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:19.552462 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74e007-a1c3-4936-ab81-dff09b961ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:19.627736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8c74e007-a1c3-4936-ab81-dff09b961ad0 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:19.629163 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8c74e007-a1c3-4936-ab81-dff09b961ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:19.755783 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 680/1378] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:25:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:19.800227 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe8a0e-3291-4960-9d7e-f78a9ab2c391 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:19.801718 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-9bfe8a0e-3291-4960-9d7e-f78a9ab2c391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:19.881325 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:19.964659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe8a0e-3291-4960-9d7e-f78a9ab2c391 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:20.098068 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef0255-6d4f-4d11-893d-b0f919e2d2a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:20.099475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef0255-6d4f-4d11-893d-b0f919e2d2a1 service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:20.099766 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef0255-6d4f-4d11-893d-b0f919e2d2a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:20.100079 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef0255-6d4f-4d11-893d-b0f919e2d2a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:20.101816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-b5ef0255-6d4f-4d11-893d-b0f919e2d2a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:20.142884 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:20.211621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef0255-6d4f-4d11-893d-b0f919e2d2a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:20.213422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef0255-6d4f-4d11-893d-b0f919e2d2a1 service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:20.234072 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 699/1379] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:25:19 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:22.366615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ac901-5533-4c24-8d9d-771433a39cef None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:22.367454 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ac901-5533-4c24-8d9d-771433a39cef None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:22.367454 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ac901-5533-4c24-8d9d-771433a39cef None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:22.368836 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 681/1380] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:25:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:25:22.374276 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b2c55-448b-492a-8a5f-134270e9a275 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:22.374857 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b2c55-448b-492a-8a5f-134270e9a275 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:22.374857 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b2c55-448b-492a-8a5f-134270e9a275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:22.450068 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-f23b2c55-448b-492a-8a5f-134270e9a275 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:22.451280 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-f23b2c55-448b-492a-8a5f-134270e9a275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:22.564878 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 700/1381] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:25:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:22.610631 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ac901-5533-4c24-8d9d-771433a39cef None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:22.612578 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f84ac901-5533-4c24-8d9d-771433a39cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:22.640752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:22.784020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ac901-5533-4c24-8d9d-771433a39cef None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:22.899061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:22.923123 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8209ec7-f16e-4cdc-b5a1-b4887d9cb1c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:22.923760 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8209ec7-f16e-4cdc-b5a1-b4887d9cb1c4 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:22.924094 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8209ec7-f16e-4cdc-b5a1-b4887d9cb1c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:22.924374 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8209ec7-f16e-4cdc-b5a1-b4887d9cb1c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:22.926033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d8209ec7-f16e-4cdc-b5a1-b4887d9cb1c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:23.025615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8209ec7-f16e-4cdc-b5a1-b4887d9cb1c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:23.027279 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8209ec7-f16e-4cdc-b5a1-b4887d9cb1c4 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:23.046785 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 682/1382] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:25:22 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:25.146029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd771829-6612-4a5e-90ed-459a47dab64d None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:25.146650 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd771829-6612-4a5e-90ed-459a47dab64d None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:25.146650 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd771829-6612-4a5e-90ed-459a47dab64d None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:25.148038 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 701/1383] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:25:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:25:25.154208 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a8300-b27f-44ec-9432-562a242b9150 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:25.154714 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a8300-b27f-44ec-9432-562a242b9150 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:25.154966 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a8300-b27f-44ec-9432-562a242b9150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:25.251773 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ec8a8300-b27f-44ec-9432-562a242b9150 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:25.253604 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ec8a8300-b27f-44ec-9432-562a242b9150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:25.322138 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:25.367207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 683/1384] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:25:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:25.415993 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd771829-6612-4a5e-90ed-459a47dab64d None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:25.417646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-dd771829-6612-4a5e-90ed-459a47dab64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:25.428545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:25.600678 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd771829-6612-4a5e-90ed-459a47dab64d None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:25.691484 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:25.725153 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f2ea6-04e3-4fda-868a-18100e302f5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:25.725787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f2ea6-04e3-4fda-868a-18100e302f5b service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:25.726111 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f2ea6-04e3-4fda-868a-18100e302f5b service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:25.726381 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f2ea6-04e3-4fda-868a-18100e302f5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:25.728095 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-679f2ea6-04e3-4fda-868a-18100e302f5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:25.825108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679f2ea6-04e3-4fda-868a-18100e302f5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:25.826855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679f2ea6-04e3-4fda-868a-18100e302f5b service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:25.849012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 702/1385] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:25:25 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:28.664580 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ad249-eb0b-4343-9e9b-d0a31ad2799d None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:28.665216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ad249-eb0b-4343-9e9b-d0a31ad2799d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:28.665216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ad249-eb0b-4343-9e9b-d0a31ad2799d None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:28.666541 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 684/1386] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:25:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:25:28.674619 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869b534-9a02-40e2-bb8b-683ea1b82924 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:28.675227 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869b534-9a02-40e2-bb8b-683ea1b82924 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:28.675361 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869b534-9a02-40e2-bb8b-683ea1b82924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:28.747860 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-7869b534-9a02-40e2-bb8b-683ea1b82924 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:28.749052 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-7869b534-9a02-40e2-bb8b-683ea1b82924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:28.801179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:28.854433 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 703/1387] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:25:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:28.904308 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ad249-eb0b-4343-9e9b-d0a31ad2799d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:28.906926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e49ad249-eb0b-4343-9e9b-d0a31ad2799d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:29.095855 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ad249-eb0b-4343-9e9b-d0a31ad2799d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:29.153900 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:29.259647 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0d236-65d3-45cd-97e9-ec0eeb39f98f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:29.260640 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0d236-65d3-45cd-97e9-ec0eeb39f98f service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:29.261019 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0d236-65d3-45cd-97e9-ec0eeb39f98f service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:29.261460 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0d236-65d3-45cd-97e9-ec0eeb39f98f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:29.264043 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5db0d236-65d3-45cd-97e9-ec0eeb39f98f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:29.379484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0d236-65d3-45cd-97e9-ec0eeb39f98f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:29.381652 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0d236-65d3-45cd-97e9-ec0eeb39f98f service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:29.400581 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 685/1388] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:25:28 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:31.512564 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5e058-4197-4841-a409-e7ee93dbce8b None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:31.513281 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5e058-4197-4841-a409-e7ee93dbce8b None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:31.513281 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5e058-4197-4841-a409-e7ee93dbce8b None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:31.514710 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 704/1389] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:25:31 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:25:31.521585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bd4de-cdf9-4281-bf08-2feb5130cc7f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:31.522192 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bd4de-cdf9-4281-bf08-2feb5130cc7f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:31.522192 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bd4de-cdf9-4281-bf08-2feb5130cc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:31.601829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:31.610668 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-5d9bd4de-cdf9-4281-bf08-2feb5130cc7f None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:31.612099 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5d9bd4de-cdf9-4281-bf08-2feb5130cc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:31.714614 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 686/1390] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:25:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:31.760156 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5e058-4197-4841-a409-e7ee93dbce8b None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:31.761794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-2ac5e058-4197-4841-a409-e7ee93dbce8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:31.934175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5e058-4197-4841-a409-e7ee93dbce8b None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:31.966220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:32.055849 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07009bf1-d167-43f8-99a5-5f7a3878abd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:32.056504 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07009bf1-d167-43f8-99a5-5f7a3878abd1 service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:32.056769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07009bf1-d167-43f8-99a5-5f7a3878abd1 service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:32.057050 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07009bf1-d167-43f8-99a5-5f7a3878abd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:32.058688 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-07009bf1-d167-43f8-99a5-5f7a3878abd1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:32.160108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07009bf1-d167-43f8-99a5-5f7a3878abd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:32.161788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07009bf1-d167-43f8-99a5-5f7a3878abd1 service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:32.184106 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 705/1391] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:25:31 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:32.481899 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51b0c4-fe0b-42a0-9d8a-117d07248cf2 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:32.482160 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51b0c4-fe0b-42a0-9d8a-117d07248cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:32.482361 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51b0c4-fe0b-42a0-9d8a-117d07248cf2 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:32.484065 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 687/1392] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:25:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:25:32.490183 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020853e-004c-4530-a763-c14e9f5ed8ce None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:32.490468 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020853e-004c-4530-a763-c14e9f5ed8ce None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:32.490769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020853e-004c-4530-a763-c14e9f5ed8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:32.541005 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:32.567173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-9020853e-004c-4530-a763-c14e9f5ed8ce None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:32.568846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-9020853e-004c-4530-a763-c14e9f5ed8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:32.679569 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 706/1393] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:25:32 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:32.737345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae1b8a-e4cc-4edf-83c7-56ad2cd3844a None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:32.737867 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae1b8a-e4cc-4edf-83c7-56ad2cd3844a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:32.737867 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae1b8a-e4cc-4edf-83c7-56ad2cd3844a None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:32.739306 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 688/1394] 10.222.0.121 () {54 vars in 971 bytes} [Thu Aug 4 10:25:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:25:32.744858 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae50de-ab13-4a73-a0f4-3b546af392e6 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:32.745151 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae50de-ab13-4a73-a0f4-3b546af392e6 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:32.745463 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae50de-ab13-4a73-a0f4-3b546af392e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:32.834376 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-ddae50de-ab13-4a73-a0f4-3b546af392e6 None None] MFA Rules not processed for user `1b31c86bc9334d2b8acc35a035f90442`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:32.835873 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-ddae50de-ab13-4a73-a0f4-3b546af392e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:32.892159 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:32.919052 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 707/1395] 10.222.0.121 () {58 vars in 1068 bytes} [Thu Aug 4 10:25:32 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:32.927194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdae1b8a-e4cc-4edf-83c7-56ad2cd3844a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:32.928867 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cdae1b8a-e4cc-4edf-83c7-56ad2cd3844a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:33.039607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdae1b8a-e4cc-4edf-83c7-56ad2cd3844a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:33.068395 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:33.120289 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0904da6-bf28-4c8a-be51-6b4343686002 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b31c86bc9334d2b8acc35a035f90442', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:33.121096 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0904da6-bf28-4c8a-be51-6b4343686002 service glance] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:33.121402 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0904da6-bf28-4c8a-be51-6b4343686002 service glance] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:33.121647 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0904da6-bf28-4c8a-be51-6b4343686002 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:33.123432 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d0904da6-bf28-4c8a-be51-6b4343686002 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:33.242889 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0904da6-bf28-4c8a-be51-6b4343686002 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:33.245231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0904da6-bf28-4c8a-be51-6b4343686002 service glance] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:33.266047 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 689/1396] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:25:32 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:34.033935 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5627d4c-e596-4070-bdb4-a054866ca222 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:34.034483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5627d4c-e596-4070-bdb4-a054866ca222 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:34.034606 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5627d4c-e596-4070-bdb4-a054866ca222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:34.108005 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-e5627d4c-e596-4070-bdb4-a054866ca222 None None] MFA Rules not processed for user `efe6e59228bd49bfbff25a11eb682426`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:34.110398 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-e5627d4c-e596-4070-bdb4-a054866ca222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:34.209242 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 708/1397] 10.222.0.121 () {58 vars in 1014 bytes} [Thu Aug 4 10:25:34 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:34.257971 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb58fc-f1b5-46c9-97e9-5b74abd59389 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:34.258525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb58fc-f1b5-46c9-97e9-5b74abd59389 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:34.258626 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb58fc-f1b5-46c9-97e9-5b74abd59389 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:34.260218 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 690/1398] 10.222.0.121 () {54 vars in 948 bytes} [Thu Aug 4 10:25:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:25:34.267993 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450cfea-980d-4730-b23d-3a358b644389 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:34.268288 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450cfea-980d-4730-b23d-3a358b644389 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:34.268568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450cfea-980d-4730-b23d-3a358b644389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:34.282433 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b37a6-cce1-40fa-a1f2-387fa9b62fce None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:34.282636 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b37a6-cce1-40fa-a1f2-387fa9b62fce None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:34.282891 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b37a6-cce1-40fa-a1f2-387fa9b62fce None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:34.284335 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 691/1399] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:25:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:25:34.289560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbec1c9-09f3-4034-b15e-2664fc875449 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:34.289852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbec1c9-09f3-4034-b15e-2664fc875449 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:34.290132 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbec1c9-09f3-4034-b15e-2664fc875449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:34.357150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-6450cfea-980d-4730-b23d-3a358b644389 None None] MFA Rules not processed for user `e16e26784601425c9883acd91f65becc`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:34.358398 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6450cfea-980d-4730-b23d-3a358b644389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:34.370032 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-dcbec1c9-09f3-4034-b15e-2664fc875449 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:34.371314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-dcbec1c9-09f3-4034-b15e-2664fc875449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:34.372537 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:34.380776 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:34.475657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 692/1400] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:25:34 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:25:34.477848 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 709/1401] 10.222.0.121 () {58 vars in 1045 bytes} [Thu Aug 4 10:25:34 2022] POST /identity/v3/auth/tokens => generated 3314 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:34.488022 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbec1c9-09f3-4034-b15e-2664fc875449 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:34.489293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-dcbec1c9-09f3-4034-b15e-2664fc875449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:34.523430 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6450cfea-980d-4730-b23d-3a358b644389 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:34.525353 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6450cfea-980d-4730-b23d-3a358b644389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:34.606563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbec1c9-09f3-4034-b15e-2664fc875449 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:34.630627 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6450cfea-980d-4730-b23d-3a358b644389 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:34.649576 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:34.670881 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:34.722967 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4613665-d613-4e1f-8225-adb8b55eaf27 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e16e26784601425c9883acd91f65becc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:34.723735 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4613665-d613-4e1f-8225-adb8b55eaf27 service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:34.724197 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4613665-d613-4e1f-8225-adb8b55eaf27 service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:34.724494 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4613665-d613-4e1f-8225-adb8b55eaf27 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:34.726367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a4613665-d613-4e1f-8225-adb8b55eaf27 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:34.762769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c576379-e056-4d1f-bbee-e1a4751ed6a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:34.763404 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c576379-e056-4d1f-bbee-e1a4751ed6a8 service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:34.763684 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c576379-e056-4d1f-bbee-e1a4751ed6a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:34.763979 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c576379-e056-4d1f-bbee-e1a4751ed6a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:34.766380 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-2c576379-e056-4d1f-bbee-e1a4751ed6a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:34.819923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4613665-d613-4e1f-8225-adb8b55eaf27 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:34.821668 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4613665-d613-4e1f-8225-adb8b55eaf27 service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:34.830217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 693/1402] 10.222.0.121 () {60 vars in 1389 bytes} [Thu Aug 4 10:25:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:25:34.862859 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c576379-e056-4d1f-bbee-e1a4751ed6a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:34.864686 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c576379-e056-4d1f-bbee-e1a4751ed6a8 service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:34.884995 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 710/1403] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:25:34 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:37.055211 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7731d-21a4-459b-b47d-29f23e81df51 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:37.055982 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7731d-21a4-459b-b47d-29f23e81df51 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:37.055982 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7731d-21a4-459b-b47d-29f23e81df51 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:37.057558 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 694/1404] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:25:37 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:25:37.063603 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfef5c0-2a57-4f37-910c-722e85cf13c5 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:37.064263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfef5c0-2a57-4f37-910c-722e85cf13c5 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:37.064263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfef5c0-2a57-4f37-910c-722e85cf13c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:37.120582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:37.144521 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-8cfef5c0-2a57-4f37-910c-722e85cf13c5 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:37.145959 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-8cfef5c0-2a57-4f37-910c-722e85cf13c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:37.256650 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 711/1405] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:25:37 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:37.303699 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7731d-21a4-459b-b47d-29f23e81df51 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:37.305365 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f3d7731d-21a4-459b-b47d-29f23e81df51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:37.389093 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:37.474453 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7731d-21a4-459b-b47d-29f23e81df51 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:37.619066 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a25d1-5bf4-4623-a3a9-ba9fa07c5074 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:37.619713 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a25d1-5bf4-4623-a3a9-ba9fa07c5074 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:37.620029 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a25d1-5bf4-4623-a3a9-ba9fa07c5074 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:37.620314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a25d1-5bf4-4623-a3a9-ba9fa07c5074 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:37.622020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f91a25d1-5bf4-4623-a3a9-ba9fa07c5074 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:37.667541 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:37.749112 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a25d1-5bf4-4623-a3a9-ba9fa07c5074 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:37.752872 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a25d1-5bf4-4623-a3a9-ba9fa07c5074 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:37.783699 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 695/1406] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:25:37 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:39.871714 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ac9c4-2307-4b42-9f20-119fa4855473 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:39.872238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ac9c4-2307-4b42-9f20-119fa4855473 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:39.872349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ac9c4-2307-4b42-9f20-119fa4855473 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:39.874030 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 712/1407] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:25:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:25:39.879412 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b7c3b-eea3-4c92-a74e-535280e57b8d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:39.880628 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b7c3b-eea3-4c92-a74e-535280e57b8d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:39.881108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b7c3b-eea3-4c92-a74e-535280e57b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:39.963855 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-124b7c3b-eea3-4c92-a74e-535280e57b8d None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:39.965507 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-124b7c3b-eea3-4c92-a74e-535280e57b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:40.080006 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 696/1408] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:25:39 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:25:40.128167 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ac9c4-2307-4b42-9f20-119fa4855473 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:25:40.129793 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-970ac9c4-2307-4b42-9f20-119fa4855473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:40.270421 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:40.325503 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ac9c4-2307-4b42-9f20-119fa4855473 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:25:40.509039 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f778e9-f29a-408e-ad29-57e59c269393 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:25:40.509586 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f778e9-f29a-408e-ad29-57e59c269393 service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:40.509879 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f778e9-f29a-408e-ad29-57e59c269393 service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:40.510156 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f778e9-f29a-408e-ad29-57e59c269393 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:40.511893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-b3f778e9-f29a-408e-ad29-57e59c269393 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:40.620827 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:40.640376 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f778e9-f29a-408e-ad29-57e59c269393 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:25:40.642142 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f778e9-f29a-408e-ad29-57e59c269393 service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:25:40.667776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 713/1409] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:25:40 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:25:50.075662 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:50.663761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=10 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:58.021223 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f240913-d7ed-4ff7-a947-93efa27edfd7 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:58.021886 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f240913-d7ed-4ff7-a947-93efa27edfd7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:58.021886 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f240913-d7ed-4ff7-a947-93efa27edfd7 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:58.023160 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 697/1410] 10.222.0.121 () {54 vars in 923 bytes} [Thu Aug 4 10:25:58 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:25:58.028874 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a4eac-e382-4d20-b438-2ab75f228a74 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:58.029616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a4eac-e382-4d20-b438-2ab75f228a74 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:58.029616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a4eac-e382-4d20-b438-2ab75f228a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:58.114022 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-294a4eac-e382-4d20-b438-2ab75f228a74 None None] MFA Rules not processed for user `01c7cfbe911c4378857848966f4f8d98`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:58.115473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-294a4eac-e382-4d20-b438-2ab75f228a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:58.231074 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 714/1411] 10.222.0.121 () {58 vars in 1020 bytes} [Thu Aug 4 10:25:58 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:25:58.246986 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47157d8b-88cc-454b-b8b9-b8ba2a875926 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:58.247275 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47157d8b-88cc-454b-b8b9-b8ba2a875926 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:58.247555 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47157d8b-88cc-454b-b8b9-b8ba2a875926 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:58.249157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 698/1412] 10.222.0.121 () {54 vars in 923 bytes} [Thu Aug 4 10:25:58 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:25:58.254572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ab139-4797-4a9a-a855-58a8ed30ce69 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:25:58.254841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ab139-4797-4a9a-a855-58a8ed30ce69 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:25:58.255185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ab139-4797-4a9a-a855-58a8ed30ce69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:25:58.331993 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:25:58.334940 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-d45ab139-4797-4a9a-a855-58a8ed30ce69 None None] MFA Rules not processed for user `0d947aeb749546309a8d4b7938087ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:25:58.336696 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d45ab139-4797-4a9a-a855-58a8ed30ce69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:25:58.464452 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 715/1413] 10.222.0.121 () {58 vars in 1020 bytes} [Thu Aug 4 10:25:58 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:00.999535 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b635de-9a84-4707-b41c-18bf52e48634 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:01.000185 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b635de-9a84-4707-b41c-18bf52e48634 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:01.000185 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b635de-9a84-4707-b41c-18bf52e48634 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:01.001487 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:26:00 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:26:01.007769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5eb1f-9779-4028-b739-2d68ff4def97 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:01.008395 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5eb1f-9779-4028-b739-2d68ff4def97 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:01.008395 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5eb1f-9779-4028-b739-2d68ff4def97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:01.078839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-51c5eb1f-9779-4028-b739-2d68ff4def97 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:01.080128 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-51c5eb1f-9779-4028-b739-2d68ff4def97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:01.141210 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:01.187330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:01.232156 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b635de-9a84-4707-b41c-18bf52e48634 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:01.233878 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f0b635de-9a84-4707-b41c-18bf52e48634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:01.400862 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b635de-9a84-4707-b41c-18bf52e48634 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:01.476771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:01.518833 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ba35c-1d1e-4043-a047-dfef31dfcc26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:01.519525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ba35c-1d1e-4043-a047-dfef31dfcc26 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:01.519822 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ba35c-1d1e-4043-a047-dfef31dfcc26 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:01.520106 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ba35c-1d1e-4043-a047-dfef31dfcc26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:01.522195 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cc2ba35c-1d1e-4043-a047-dfef31dfcc26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:01.621195 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ba35c-1d1e-4043-a047-dfef31dfcc26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:01.622873 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ba35c-1d1e-4043-a047-dfef31dfcc26 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:01.642442 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 700/1416] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:26:01 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:02.602881 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcece5-5373-44ec-9bda-3d081f1de3f1 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:02.603463 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcece5-5373-44ec-9bda-3d081f1de3f1 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:02.603562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcece5-5373-44ec-9bda-3d081f1de3f1 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:02.605069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 10:26:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:26:02.611029 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75316e25-2cb6-4710-baf4-3f67fa2d48d9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:02.611478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75316e25-2cb6-4710-baf4-3f67fa2d48d9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:02.611598 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75316e25-2cb6-4710-baf4-3f67fa2d48d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:02.693963 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-75316e25-2cb6-4710-baf4-3f67fa2d48d9 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:02.695195 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-75316e25-2cb6-4710-baf4-3f67fa2d48d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:02.723822 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:02.805343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 10:26:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:02.965934 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dcece5-5373-44ec-9bda-3d081f1de3f1 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:02.967432 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-a4dcece5-5373-44ec-9bda-3d081f1de3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:03.129220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dcece5-5373-44ec-9bda-3d081f1de3f1 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:03.173040 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:03.252982 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87818c8b-ab6d-49f4-95a4-39db70db76bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:03.253522 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87818c8b-ab6d-49f4-95a4-39db70db76bc service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:03.253811 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87818c8b-ab6d-49f4-95a4-39db70db76bc service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:03.254092 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87818c8b-ab6d-49f4-95a4-39db70db76bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:03.255847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-87818c8b-ab6d-49f4-95a4-39db70db76bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:03.352695 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87818c8b-ab6d-49f4-95a4-39db70db76bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:03.355237 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87818c8b-ab6d-49f4-95a4-39db70db76bc service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:03.378765 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 718/1419] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:26:02 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:04.309422 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4ff57-bab0-4eed-997c-53b791f57375 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:04.310090 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4ff57-bab0-4eed-997c-53b791f57375 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:04.310090 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4ff57-bab0-4eed-997c-53b791f57375 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:04.311223 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 10:26:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:26:04.320173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9fa29-3afd-484b-9947-1a035f17a2a3 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:04.320704 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9fa29-3afd-484b-9947-1a035f17a2a3 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:04.320704 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9fa29-3afd-484b-9947-1a035f17a2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:04.382937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:04.406327 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-e7c9fa29-3afd-484b-9947-1a035f17a2a3 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:04.407604 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-e7c9fa29-3afd-484b-9947-1a035f17a2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:04.512787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 10:26:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:04.674452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4ff57-bab0-4eed-997c-53b791f57375 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:04.676078 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-64f4ff57-bab0-4eed-997c-53b791f57375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:04.834809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4ff57-bab0-4eed-997c-53b791f57375 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:04.848011 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:04.960357 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23686af-ab1f-4ec3-98f6-e5be9f6042c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:04.961040 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23686af-ab1f-4ec3-98f6-e5be9f6042c6 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:04.961408 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23686af-ab1f-4ec3-98f6-e5be9f6042c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:04.961682 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23686af-ab1f-4ec3-98f6-e5be9f6042c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:04.963370 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f23686af-ab1f-4ec3-98f6-e5be9f6042c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:05.060220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23686af-ab1f-4ec3-98f6-e5be9f6042c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:05.061871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23686af-ab1f-4ec3-98f6-e5be9f6042c6 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:05.080468 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 703/1422] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:26:04 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:05.956226 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df86a13-52d0-4ed8-a0f5-28e73964e867 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:05.956934 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df86a13-52d0-4ed8-a0f5-28e73964e867 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:05.956934 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df86a13-52d0-4ed8-a0f5-28e73964e867 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:05.958233 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 10:26:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:26:05.963769 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2667cd-22ad-4db3-8b00-8798a21f9a47 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:05.964354 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2667cd-22ad-4db3-8b00-8798a21f9a47 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:05.964471 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2667cd-22ad-4db3-8b00-8798a21f9a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:05.986574 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:06.048650 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2b2667cd-22ad-4db3-8b00-8798a21f9a47 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:06.049892 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2b2667cd-22ad-4db3-8b00-8798a21f9a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:06.152592 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 10:26:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:06.309006 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df86a13-52d0-4ed8-a0f5-28e73964e867 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:06.310601 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6df86a13-52d0-4ed8-a0f5-28e73964e867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:06.444892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:06.486461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df86a13-52d0-4ed8-a0f5-28e73964e867 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:06.634587 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90dd61e-729e-4bfd-9975-00dc16101bca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:06.635248 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dd61e-729e-4bfd-9975-00dc16101bca service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:06.635525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dd61e-729e-4bfd-9975-00dc16101bca service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:06.635800 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dd61e-729e-4bfd-9975-00dc16101bca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:06.637620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d90dd61e-729e-4bfd-9975-00dc16101bca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:06.731368 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:06.748086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90dd61e-729e-4bfd-9975-00dc16101bca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:06.749771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90dd61e-729e-4bfd-9975-00dc16101bca service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:06.772109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 721/1425] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:26:06 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:07.646817 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028861e-4bd9-4317-a531-44a492a69431 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:07.647542 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028861e-4bd9-4317-a531-44a492a69431 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:07.647542 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028861e-4bd9-4317-a531-44a492a69431 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:07.648926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 10:26:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:26:07.654686 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666456d5-6316-4413-b9b8-4c1d99b4b8a1 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:07.655304 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666456d5-6316-4413-b9b8-4c1d99b4b8a1 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:07.655304 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666456d5-6316-4413-b9b8-4c1d99b4b8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:07.750207 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-666456d5-6316-4413-b9b8-4c1d99b4b8a1 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:07.751501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-666456d5-6316-4413-b9b8-4c1d99b4b8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:07.853510 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 10:26:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:08.006277 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028861e-4bd9-4317-a531-44a492a69431 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:08.007989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0028861e-4bd9-4317-a531-44a492a69431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:08.103860 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:08.169523 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028861e-4bd9-4317-a531-44a492a69431 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:08.288254 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b666f24-19c2-489d-880e-eb3b5c4a1d21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:08.288888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b666f24-19c2-489d-880e-eb3b5c4a1d21 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:08.289174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b666f24-19c2-489d-880e-eb3b5c4a1d21 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:08.289480 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b666f24-19c2-489d-880e-eb3b5c4a1d21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:08.291183 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0b666f24-19c2-489d-880e-eb3b5c4a1d21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:08.342858 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:08.388616 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b666f24-19c2-489d-880e-eb3b5c4a1d21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:08.390275 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b666f24-19c2-489d-880e-eb3b5c4a1d21 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:08.409358 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 706/1428] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:26:08 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:09.242275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445a348-021b-4cc2-b882-b21f4bd655ca None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:09.242975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445a348-021b-4cc2-b882-b21f4bd655ca None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:09.242975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445a348-021b-4cc2-b882-b21f4bd655ca None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:09.244256 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 10:26:09 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:26:09.249876 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d31cb-4f30-47e0-b1e6-a6b2384335c9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:09.250461 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d31cb-4f30-47e0-b1e6-a6b2384335c9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:09.250461 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d31cb-4f30-47e0-b1e6-a6b2384335c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:09.338576 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4d5d31cb-4f30-47e0-b1e6-a6b2384335c9 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:09.339836 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4d5d31cb-4f30-47e0-b1e6-a6b2384335c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:09.457916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 10:26:09 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:09.611611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445a348-021b-4cc2-b882-b21f4bd655ca None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:09.613190 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d445a348-021b-4cc2-b882-b21f4bd655ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:09.667203 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:09.785135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445a348-021b-4cc2-b882-b21f4bd655ca None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:09.911189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264df0c-85ba-4741-8bff-0bd62c72ae3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:09.911813 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264df0c-85ba-4741-8bff-0bd62c72ae3f service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:09.912134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264df0c-85ba-4741-8bff-0bd62c72ae3f service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:09.912408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264df0c-85ba-4741-8bff-0bd62c72ae3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:09.914109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c264df0c-85ba-4741-8bff-0bd62c72ae3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:09.924896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:10.024111 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264df0c-85ba-4741-8bff-0bd62c72ae3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:10.025763 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264df0c-85ba-4741-8bff-0bd62c72ae3f service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:10.046040 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 724/1431] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:26:09 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:10.873041 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130658a2-b719-4c3b-b011-099acb04ee44 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:10.873984 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130658a2-b719-4c3b-b011-099acb04ee44 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:10.873984 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130658a2-b719-4c3b-b011-099acb04ee44 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:10.876147 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 10:26:10 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:26:10.881581 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575da68b-d059-46cd-895b-84b493a5f0dc None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:10.882078 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575da68b-d059-46cd-895b-84b493a5f0dc None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:10.882184 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575da68b-d059-46cd-895b-84b493a5f0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:10.968395 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-575da68b-d059-46cd-895b-84b493a5f0dc None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:10.969644 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-575da68b-d059-46cd-895b-84b493a5f0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:11.042587 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:11.077175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 10:26:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:11.228466 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130658a2-b719-4c3b-b011-099acb04ee44 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:11.229917 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-130658a2-b719-4c3b-b011-099acb04ee44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:11.250305 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:11.394196 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130658a2-b719-4c3b-b011-099acb04ee44 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:11.505371 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:11.544669 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ea5bf-b62c-4338-be1e-e2fbcdd75ebf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:11.545390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ea5bf-b62c-4338-be1e-e2fbcdd75ebf service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:11.545761 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ea5bf-b62c-4338-be1e-e2fbcdd75ebf service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:11.546115 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ea5bf-b62c-4338-be1e-e2fbcdd75ebf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:11.548473 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-996ea5bf-b62c-4338-be1e-e2fbcdd75ebf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:11.648193 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ea5bf-b62c-4338-be1e-e2fbcdd75ebf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:11.649863 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ea5bf-b62c-4338-be1e-e2fbcdd75ebf service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:11.668699 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 709/1434] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:26:11 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:12.484480 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de24b60-00fd-4908-a403-5d03af0207f7 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:12.485133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de24b60-00fd-4908-a403-5d03af0207f7 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:12.485133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de24b60-00fd-4908-a403-5d03af0207f7 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:12.486445 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 10:26:12 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:26:12.492885 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc074d-bf13-4d76-886e-8e35d435dcd7 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:12.493727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc074d-bf13-4d76-886e-8e35d435dcd7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:12.493727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc074d-bf13-4d76-886e-8e35d435dcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:12.574575 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c4bc074d-bf13-4d76-886e-8e35d435dcd7 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:12.575826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c4bc074d-bf13-4d76-886e-8e35d435dcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:12.618759 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:12.676532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 10:26:12 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:12.836456 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de24b60-00fd-4908-a403-5d03af0207f7 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:12.838199 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-3de24b60-00fd-4908-a403-5d03af0207f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:13.001509 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de24b60-00fd-4908-a403-5d03af0207f7 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:13.061417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:13.139532 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324fca5-95be-4f43-84bb-ca0e7a187a32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:13.140099 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324fca5-95be-4f43-84bb-ca0e7a187a32 service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:13.140391 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324fca5-95be-4f43-84bb-ca0e7a187a32 service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:13.140673 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324fca5-95be-4f43-84bb-ca0e7a187a32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:13.142605 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-0324fca5-95be-4f43-84bb-ca0e7a187a32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:13.256790 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324fca5-95be-4f43-84bb-ca0e7a187a32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:13.258433 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324fca5-95be-4f43-84bb-ca0e7a187a32 service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:13.278381 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 727/1437] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:26:12 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:14.119873 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702b019-1969-4283-8ce4-d4962447b35c None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:14.120518 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702b019-1969-4283-8ce4-d4962447b35c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:14.120518 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702b019-1969-4283-8ce4-d4962447b35c None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:14.122638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 10:26:14 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:26:14.130561 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f81163-9a24-4369-808d-94e9de17ae92 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:14.131151 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f81163-9a24-4369-808d-94e9de17ae92 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:14.131151 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f81163-9a24-4369-808d-94e9de17ae92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:14.211986 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-f0f81163-9a24-4369-808d-94e9de17ae92 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:14.213819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-f0f81163-9a24-4369-808d-94e9de17ae92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:14.217057 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:14.322778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 10:26:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:14.477565 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702b019-1969-4283-8ce4-d4962447b35c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:14.479069 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e702b019-1969-4283-8ce4-d4962447b35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:14.645561 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702b019-1969-4283-8ce4-d4962447b35c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:14.679969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:14.781550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d504e-636e-4df4-ba6c-2d7cf0d38e74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:14.782107 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d504e-636e-4df4-ba6c-2d7cf0d38e74 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:14.782384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d504e-636e-4df4-ba6c-2d7cf0d38e74 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:14.782658 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d504e-636e-4df4-ba6c-2d7cf0d38e74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:14.784392 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3f8d504e-636e-4df4-ba6c-2d7cf0d38e74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:14.884235 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d504e-636e-4df4-ba6c-2d7cf0d38e74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:14.885894 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d504e-636e-4df4-ba6c-2d7cf0d38e74 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:14.905455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 712/1440] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:26:14 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:16.827804 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8389e69-a492-4c63-89e9-d657c75fb9de None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:16.828807 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8389e69-a492-4c63-89e9-d657c75fb9de None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:16.828807 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8389e69-a492-4c63-89e9-d657c75fb9de None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:16.830755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:26:16 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:26:16.837620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a01f4-02c3-4de5-9f25-8d8462df9dbf None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:16.838081 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a01f4-02c3-4de5-9f25-8d8462df9dbf None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:16.838185 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a01f4-02c3-4de5-9f25-8d8462df9dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:16.884630 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:16.916325 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e70a01f4-02c3-4de5-9f25-8d8462df9dbf None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:16.917632 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e70a01f4-02c3-4de5-9f25-8d8462df9dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:17.028110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:26:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:17.078788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8389e69-a492-4c63-89e9-d657c75fb9de None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:17.080481 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c8389e69-a492-4c63-89e9-d657c75fb9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:17.199712 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8389e69-a492-4c63-89e9-d657c75fb9de None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:17.240236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:17.295488 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f8a8c-c7ad-4047-a50e-390fed1211b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:17.296175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f8a8c-c7ad-4047-a50e-390fed1211b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:17.296460 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f8a8c-c7ad-4047-a50e-390fed1211b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:17.296737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f8a8c-c7ad-4047-a50e-390fed1211b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:17.298430 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-8b1f8a8c-c7ad-4047-a50e-390fed1211b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:17.425862 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f8a8c-c7ad-4047-a50e-390fed1211b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:17.427367 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f8a8c-c7ad-4047-a50e-390fed1211b4 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:17.446463 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 730/1443] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:26:17 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:19.666511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9afe36-133c-4c6c-ad23-60223f52e4dc None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:19.667152 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9afe36-133c-4c6c-ad23-60223f52e4dc None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:19.667152 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9afe36-133c-4c6c-ad23-60223f52e4dc None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:19.669169 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:26:19 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:26:19.675647 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583c51c-6ac5-40c3-9a1d-d1101bf53620 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:19.676158 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583c51c-6ac5-40c3-9a1d-d1101bf53620 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:19.676281 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583c51c-6ac5-40c3-9a1d-d1101bf53620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:19.753665 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-8583c51c-6ac5-40c3-9a1d-d1101bf53620 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:19.754985 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-8583c51c-6ac5-40c3-9a1d-d1101bf53620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:19.768722 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:19.866338 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:26:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:19.909020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9afe36-133c-4c6c-ad23-60223f52e4dc None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:19.910621 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cb9afe36-133c-4c6c-ad23-60223f52e4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:20.037652 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9afe36-133c-4c6c-ad23-60223f52e4dc None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:20.050345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:20.125397 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95617af0-c973-4943-ac9f-69adfe6da9a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:20.125940 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95617af0-c973-4943-ac9f-69adfe6da9a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:20.126223 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95617af0-c973-4943-ac9f-69adfe6da9a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:20.126501 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95617af0-c973-4943-ac9f-69adfe6da9a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:20.128225 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-95617af0-c973-4943-ac9f-69adfe6da9a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:20.254941 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95617af0-c973-4943-ac9f-69adfe6da9a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:20.256478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95617af0-c973-4943-ac9f-69adfe6da9a9 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:20.276891 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 715/1446] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:26:19 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:22.744888 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78d01c-f3b5-4109-96fb-72e6e9d8ca6d None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:22.745587 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78d01c-f3b5-4109-96fb-72e6e9d8ca6d None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:22.745587 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78d01c-f3b5-4109-96fb-72e6e9d8ca6d None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:22.747468 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:26:22 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:26:22.755848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d695a4-1f7b-4a77-ad88-a129ba22db75 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:22.756368 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d695a4-1f7b-4a77-ad88-a129ba22db75 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:22.756484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d695a4-1f7b-4a77-ad88-a129ba22db75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:22.812353 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:22.832939 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-26d695a4-1f7b-4a77-ad88-a129ba22db75 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:22.834299 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-26d695a4-1f7b-4a77-ad88-a129ba22db75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:22.934097 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:26:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:22.978933 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78d01c-f3b5-4109-96fb-72e6e9d8ca6d None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:22.981485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-cb78d01c-f3b5-4109-96fb-72e6e9d8ca6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:23.095068 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78d01c-f3b5-4109-96fb-72e6e9d8ca6d None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:23.139133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:23.177544 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d7046-667d-49cf-a3ba-9996d9c8f25f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:23.178188 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d7046-667d-49cf-a3ba-9996d9c8f25f service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:23.178471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d7046-667d-49cf-a3ba-9996d9c8f25f service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:23.178743 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d7046-667d-49cf-a3ba-9996d9c8f25f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:23.180497 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-a34d7046-667d-49cf-a3ba-9996d9c8f25f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:23.309874 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34d7046-667d-49cf-a3ba-9996d9c8f25f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:23.311366 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34d7046-667d-49cf-a3ba-9996d9c8f25f service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:23.331461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 733/1449] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:26:22 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:25.509383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f976ff-8dd7-4ab0-a47b-b58d912ca7d3 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:25.510398 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f976ff-8dd7-4ab0-a47b-b58d912ca7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:25.510398 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f976ff-8dd7-4ab0-a47b-b58d912ca7d3 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:25.512851 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:26:25 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:26:25.522089 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61b4e9-8ff3-4acc-aee7-61fb77fd55f7 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:25.522903 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61b4e9-8ff3-4acc-aee7-61fb77fd55f7 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:25.523093 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61b4e9-8ff3-4acc-aee7-61fb77fd55f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:25.609578 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-2d61b4e9-8ff3-4acc-aee7-61fb77fd55f7 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:25.610855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-2d61b4e9-8ff3-4acc-aee7-61fb77fd55f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:25.633358 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:25.727782 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:26:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:25.768634 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f976ff-8dd7-4ab0-a47b-b58d912ca7d3 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:25.770389 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-63f976ff-8dd7-4ab0-a47b-b58d912ca7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:25.894342 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f976ff-8dd7-4ab0-a47b-b58d912ca7d3 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:25.922357 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:25.980722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f5896-f371-4df0-b3f4-8d908a44753e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:25.981354 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f5896-f371-4df0-b3f4-8d908a44753e service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:25.981635 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f5896-f371-4df0-b3f4-8d908a44753e service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:25.981916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f5896-f371-4df0-b3f4-8d908a44753e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:25.983913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-837f5896-f371-4df0-b3f4-8d908a44753e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:26.106037 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f5896-f371-4df0-b3f4-8d908a44753e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:26.107513 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f5896-f371-4df0-b3f4-8d908a44753e service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:26.131125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 718/1452] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:26:25 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:28.351877 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa8acf-cd70-4e2a-a34f-ec8e4159b587 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:28.352878 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa8acf-cd70-4e2a-a34f-ec8e4159b587 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:28.352878 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa8acf-cd70-4e2a-a34f-ec8e4159b587 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:28.355257 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:26:28 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:26:28.361439 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039489b6-e079-4803-82eb-5107b44717fd None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:28.361960 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039489b6-e079-4803-82eb-5107b44717fd None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:28.361960 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039489b6-e079-4803-82eb-5107b44717fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:28.436027 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:28.439097 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-039489b6-e079-4803-82eb-5107b44717fd None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:28.441201 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-039489b6-e079-4803-82eb-5107b44717fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:28.557078 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:26:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:28.601004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa8acf-cd70-4e2a-a34f-ec8e4159b587 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:28.602676 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-8aaa8acf-cd70-4e2a-a34f-ec8e4159b587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:28.730194 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa8acf-cd70-4e2a-a34f-ec8e4159b587 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:28.783371 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:28.812576 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ed133-2501-4f6d-b586-de69ed9d59b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:28.813222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ed133-2501-4f6d-b586-de69ed9d59b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:28.813511 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ed133-2501-4f6d-b586-de69ed9d59b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:28.813792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ed133-2501-4f6d-b586-de69ed9d59b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:28.815476 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-cc4ed133-2501-4f6d-b586-de69ed9d59b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:28.931263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ed133-2501-4f6d-b586-de69ed9d59b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:28.932694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ed133-2501-4f6d-b586-de69ed9d59b5 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:28.951829 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 736/1455] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:26:28 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:31.142926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788139f8-1a16-4eb1-b165-922e0dba913b None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:31.143584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788139f8-1a16-4eb1-b165-922e0dba913b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:31.143584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788139f8-1a16-4eb1-b165-922e0dba913b None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:31.145054 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:26:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:26:31.150893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d85ba-6abb-4b06-a69e-92636c9883ad None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:31.151381 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d85ba-6abb-4b06-a69e-92636c9883ad None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:31.151482 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d85ba-6abb-4b06-a69e-92636c9883ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:31.223134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-e91d85ba-6abb-4b06-a69e-92636c9883ad None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:31.224448 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-e91d85ba-6abb-4b06-a69e-92636c9883ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:31.257380 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:31.332799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:26:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:31.381482 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788139f8-1a16-4eb1-b165-922e0dba913b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:31.383095 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-788139f8-1a16-4eb1-b165-922e0dba913b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:31.511526 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788139f8-1a16-4eb1-b165-922e0dba913b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:31.545792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:31.593891 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102a8dc-a98d-4a2b-abcb-1f9079e464b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:31.594532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102a8dc-a98d-4a2b-abcb-1f9079e464b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:31.594818 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102a8dc-a98d-4a2b-abcb-1f9079e464b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:31.595085 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102a8dc-a98d-4a2b-abcb-1f9079e464b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:31.596811 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7102a8dc-a98d-4a2b-abcb-1f9079e464b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:31.724202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102a8dc-a98d-4a2b-abcb-1f9079e464b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:31.725671 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102a8dc-a98d-4a2b-abcb-1f9079e464b5 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:31.747019 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 721/1458] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:26:31 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:33.883347 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958cc4e-457f-407b-8272-d4c843b296c8 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:33.884304 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958cc4e-457f-407b-8272-d4c843b296c8 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:33.884304 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958cc4e-457f-407b-8272-d4c843b296c8 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:33.885922 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:26:33 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:26:33.893849 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7bf4e-7031-4e1e-ab2d-4b094f617f55 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:33.894312 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7bf4e-7031-4e1e-ab2d-4b094f617f55 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:33.894644 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7bf4e-7031-4e1e-ab2d-4b094f617f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:34.098167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-3fd7bf4e-7031-4e1e-ab2d-4b094f617f55 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:34.099438 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3fd7bf4e-7031-4e1e-ab2d-4b094f617f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:34.109346 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:34.201662 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:26:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:34.242387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2958cc4e-457f-407b-8272-d4c843b296c8 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:34.244049 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-2958cc4e-457f-407b-8272-d4c843b296c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:34.368579 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2958cc4e-457f-407b-8272-d4c843b296c8 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:34.434020 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:34.454364 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad1de5-a876-4220-924c-572b31273259 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:34.454927 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad1de5-a876-4220-924c-572b31273259 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:34.455250 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad1de5-a876-4220-924c-572b31273259 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:34.455477 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad1de5-a876-4220-924c-572b31273259 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:34.457201 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c6ad1de5-a876-4220-924c-572b31273259 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:34.589358 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ad1de5-a876-4220-924c-572b31273259 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:34.590846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ad1de5-a876-4220-924c-572b31273259 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:34.611276 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 739/1461] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:26:34 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:36.729277 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b450f28-1233-49fa-8e51-ed8a0e1e6904 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:36.729986 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b450f28-1233-49fa-8e51-ed8a0e1e6904 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:36.729986 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b450f28-1233-49fa-8e51-ed8a0e1e6904 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:36.731208 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:26:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:26:36.737453 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b107e-5c8c-4c84-bb7d-f536608db97a None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:36.737977 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b107e-5c8c-4c84-bb7d-f536608db97a None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:36.737977 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b107e-5c8c-4c84-bb7d-f536608db97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:36.807133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-022b107e-5c8c-4c84-bb7d-f536608db97a None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:36.808310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-022b107e-5c8c-4c84-bb7d-f536608db97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:36.849884 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:36.914805 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:26:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:36.953504 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b450f28-1233-49fa-8e51-ed8a0e1e6904 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:36.955100 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6b450f28-1233-49fa-8e51-ed8a0e1e6904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:37.090817 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b450f28-1233-49fa-8e51-ed8a0e1e6904 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:37.135088 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:37.173802 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a2645-e63a-4d7d-9772-44bf2506932f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:37.174444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a2645-e63a-4d7d-9772-44bf2506932f service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:37.174719 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a2645-e63a-4d7d-9772-44bf2506932f service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:37.174994 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a2645-e63a-4d7d-9772-44bf2506932f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:37.176770 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-877a2645-e63a-4d7d-9772-44bf2506932f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:37.295587 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a2645-e63a-4d7d-9772-44bf2506932f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:37.297300 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a2645-e63a-4d7d-9772-44bf2506932f service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:37.317466 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 724/1464] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:26:36 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:39.367996 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafe1ed-b826-4a73-96c9-9fb6f97127c7 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:39.369161 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafe1ed-b826-4a73-96c9-9fb6f97127c7 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:39.369161 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafe1ed-b826-4a73-96c9-9fb6f97127c7 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:39.371305 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:26:39 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:26:39.388526 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7061b6-dbe0-4283-a8cb-c1d4efaf9516 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:39.389502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7061b6-dbe0-4283-a8cb-c1d4efaf9516 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:39.389502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7061b6-dbe0-4283-a8cb-c1d4efaf9516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:39.468691 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-5d7061b6-dbe0-4283-a8cb-c1d4efaf9516 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:39.469863 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5d7061b6-dbe0-4283-a8cb-c1d4efaf9516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:39.491440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:39.575842 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:26:39 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:39.617985 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafe1ed-b826-4a73-96c9-9fb6f97127c7 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:39.619646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-dbafe1ed-b826-4a73-96c9-9fb6f97127c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:39.732365 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafe1ed-b826-4a73-96c9-9fb6f97127c7 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:39.809978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:39.817694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a2927-a9ae-4fc3-932f-c08eddac7105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:39.818329 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a2927-a9ae-4fc3-932f-c08eddac7105 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:39.818635 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a2927-a9ae-4fc3-932f-c08eddac7105 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:39.818919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a2927-a9ae-4fc3-932f-c08eddac7105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:39.820629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-7f4a2927-a9ae-4fc3-932f-c08eddac7105 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:39.942549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a2927-a9ae-4fc3-932f-c08eddac7105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:39.944023 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a2927-a9ae-4fc3-932f-c08eddac7105 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:39.963485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 742/1467] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:26:39 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:42.007753 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb622cee-e622-40e5-ad3c-479284592cb4 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:42.008828 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb622cee-e622-40e5-ad3c-479284592cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:42.008828 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb622cee-e622-40e5-ad3c-479284592cb4 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:42.011409 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:26:42 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:26:42.019354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c227f-3aa1-49d7-97bd-8231e68a935d None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:42.020376 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c227f-3aa1-49d7-97bd-8231e68a935d None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:42.020376 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c227f-3aa1-49d7-97bd-8231e68a935d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:42.098896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-a93c227f-3aa1-49d7-97bd-8231e68a935d None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:26:42.100214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-a93c227f-3aa1-49d7-97bd-8231e68a935d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:42.155151 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:42.207490 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:26:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:26:42.249378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb622cee-e622-40e5-ad3c-479284592cb4 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:26:42.251818 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cb622cee-e622-40e5-ad3c-479284592cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:42.369569 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb622cee-e622-40e5-ad3c-479284592cb4 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:26:42.422778 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:42.452029 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede1f3f-b033-4b20-8f4c-2f21bc683846 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:26:42.452698 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede1f3f-b033-4b20-8f4c-2f21bc683846 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:26:42.452972 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede1f3f-b033-4b20-8f4c-2f21bc683846 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:26:42.453247 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede1f3f-b033-4b20-8f4c-2f21bc683846 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:26:42.455002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-aede1f3f-b033-4b20-8f4c-2f21bc683846 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:26:42.570165 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede1f3f-b033-4b20-8f4c-2f21bc683846 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:26:42.571779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede1f3f-b033-4b20-8f4c-2f21bc683846 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:26:42.590973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 727/1470] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:26:42 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:26:52.205503 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=22 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:26:52.588302 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=66 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:27:02.794241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428289e-b916-41bc-8393-0d857c45c368 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:02.794910 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428289e-b916-41bc-8393-0d857c45c368 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:02.794910 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428289e-b916-41bc-8393-0d857c45c368 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:02.796248 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 744/1471] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:27:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:27:02.801928 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761acb31-8a8a-4709-accc-fa335120e6d6 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:02.802394 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761acb31-8a8a-4709-accc-fa335120e6d6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:02.802521 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761acb31-8a8a-4709-accc-fa335120e6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:02.874846 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-761acb31-8a8a-4709-accc-fa335120e6d6 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:27:02.876054 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-761acb31-8a8a-4709-accc-fa335120e6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:02.987935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 728/1472] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:27:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:27:03.078752 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bad2f-3ff1-49ed-a70d-5cb0e873834d None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:03.079040 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bad2f-3ff1-49ed-a70d-5cb0e873834d None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:03.079347 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bad2f-3ff1-49ed-a70d-5cb0e873834d None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:03.080756 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 745/1473] 10.222.0.121 () {54 vars in 971 bytes} [Thu Aug 4 10:27:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:27:03.086508 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3904ec4-47e9-4a3e-9749-248bb7a9e00d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:03.086842 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3904ec4-47e9-4a3e-9749-248bb7a9e00d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:03.087175 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3904ec4-47e9-4a3e-9749-248bb7a9e00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:03.171914 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c3904ec4-47e9-4a3e-9749-248bb7a9e00d None None] MFA Rules not processed for user `1b31c86bc9334d2b8acc35a035f90442`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:27:03.173106 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c3904ec4-47e9-4a3e-9749-248bb7a9e00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:03.195726 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:27:03.252831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 729/1474] 10.222.0.121 () {58 vars in 1068 bytes} [Thu Aug 4 10:27:03 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:27:03.265038 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5bad2f-3ff1-49ed-a70d-5cb0e873834d None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:03.267465 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-5c5bad2f-3ff1-49ed-a70d-5cb0e873834d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:03.382343 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5bad2f-3ff1-49ed-a70d-5cb0e873834d None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:03.459898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344808a-cbbb-4242-9e53-77f55482f191 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b31c86bc9334d2b8acc35a035f90442', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:03.460468 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344808a-cbbb-4242-9e53-77f55482f191 service glance] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:03.460749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344808a-cbbb-4242-9e53-77f55482f191 service glance] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:03.461010 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344808a-cbbb-4242-9e53-77f55482f191 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:03.462676 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-7344808a-cbbb-4242-9e53-77f55482f191 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:03.656811 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:27:03.715811 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344808a-cbbb-4242-9e53-77f55482f191 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:03.717209 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344808a-cbbb-4242-9e53-77f55482f191 service glance] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:03.736358 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 746/1475] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:27:03 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:27:05.919749 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af1d43-2dc6-4ed0-b2e5-39a7dbe47736 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:05.920318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af1d43-2dc6-4ed0-b2e5-39a7dbe47736 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:05.920318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af1d43-2dc6-4ed0-b2e5-39a7dbe47736 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:05.921859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 730/1476] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:27:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:27:05.928074 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e129d9-8ab8-412f-9510-58c1869d3ede None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:05.928640 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e129d9-8ab8-412f-9510-58c1869d3ede None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:05.928640 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e129d9-8ab8-412f-9510-58c1869d3ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:06.006704 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-17e129d9-8ab8-412f-9510-58c1869d3ede None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:27:06.007986 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-17e129d9-8ab8-412f-9510-58c1869d3ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:06.110550 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 747/1477] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:27:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:27:06.134419 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af1d43-2dc6-4ed0-b2e5-39a7dbe47736 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:06.135991 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-74af1d43-2dc6-4ed0-b2e5-39a7dbe47736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:06.270525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af1d43-2dc6-4ed0-b2e5-39a7dbe47736 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:06.323650 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:27:06.371127 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9591ef-c207-4727-a551-b223edeb6694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:06.371762 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9591ef-c207-4727-a551-b223edeb6694 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:06.372081 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9591ef-c207-4727-a551-b223edeb6694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:06.372362 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9591ef-c207-4727-a551-b223edeb6694 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:06.373082 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9591ef-c207-4727-a551-b223edeb6694 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:06.374911 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: WARNING py.warnings [None req-ad9591ef-c207-4727-a551-b223edeb6694 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:27:06.374911 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: warnings.warn(msg) Aug 04 10:27:06.374911 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: Aug 04 10:27:06.376091 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9591ef-c207-4727-a551-b223edeb6694 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:06.397486 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 731/1478] 10.222.0.121 () {56 vars in 1113 bytes} [Thu Aug 4 10:27:06 2022] GET /identity/v3/projects => generated 2562 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:27:08.524397 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af973a4d-4aee-4853-8dc1-37e4a328423a None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:08.525088 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af973a4d-4aee-4853-8dc1-37e4a328423a None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:08.525088 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af973a4d-4aee-4853-8dc1-37e4a328423a None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:08.527013 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 748/1479] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:27:08 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:27:08.536031 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7808f99-dba7-4068-869a-0ca6d243b727 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:08.536838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7808f99-dba7-4068-869a-0ca6d243b727 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:08.536838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7808f99-dba7-4068-869a-0ca6d243b727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:08.621564 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b7808f99-dba7-4068-869a-0ca6d243b727 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:27:08.622694 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b7808f99-dba7-4068-869a-0ca6d243b727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:08.734913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 732/1480] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:27:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:27:08.789317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af973a4d-4aee-4853-8dc1-37e4a328423a None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:08.791092 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-af973a4d-4aee-4853-8dc1-37e4a328423a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:08.802006 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:27:08.951830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af973a4d-4aee-4853-8dc1-37e4a328423a None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:09.039705 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:27:09.074591 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a533ae-cfe2-4838-b106-55b99175eed4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:09.075226 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a533ae-cfe2-4838-b106-55b99175eed4 service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:09.075499 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a533ae-cfe2-4838-b106-55b99175eed4 service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:09.075777 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a533ae-cfe2-4838-b106-55b99175eed4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:09.077450 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-07a533ae-cfe2-4838-b106-55b99175eed4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:09.185642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a533ae-cfe2-4838-b106-55b99175eed4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:09.187278 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a533ae-cfe2-4838-b106-55b99175eed4 service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:09.208172 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 749/1481] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:27:08 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:27:11.530779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02a98f-e470-4c7d-b916-e489cffe2459 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:11.531496 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02a98f-e470-4c7d-b916-e489cffe2459 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:11.531496 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02a98f-e470-4c7d-b916-e489cffe2459 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:11.532742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 733/1482] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:27:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:27:11.538586 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347f44f-dd27-49f8-b811-1675e0a2adfd None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:11.539163 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347f44f-dd27-49f8-b811-1675e0a2adfd None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:11.539163 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347f44f-dd27-49f8-b811-1675e0a2adfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:11.614818 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-8347f44f-dd27-49f8-b811-1675e0a2adfd None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:27:11.616111 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-8347f44f-dd27-49f8-b811-1675e0a2adfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:11.668851 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:27:11.720776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 750/1483] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:27:11 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:27:11.758485 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02a98f-e470-4c7d-b916-e489cffe2459 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:11.760399 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1e02a98f-e470-4c7d-b916-e489cffe2459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:11.787825 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:27:11.892560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02a98f-e470-4c7d-b916-e489cffe2459 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:11.974880 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e4f36-4531-4522-8809-d66bb95c001f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:11.975521 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e4f36-4531-4522-8809-d66bb95c001f service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:11.975814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e4f36-4531-4522-8809-d66bb95c001f service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:11.976127 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e4f36-4531-4522-8809-d66bb95c001f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:11.977831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-398e4f36-4531-4522-8809-d66bb95c001f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:12.057482 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:27:12.095260 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e4f36-4531-4522-8809-d66bb95c001f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:12.096827 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e4f36-4531-4522-8809-d66bb95c001f service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:12.117244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 734/1484] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:27:11 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:27:14.227196 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d9faa-36dd-4c9d-8c5d-789e6c0029e3 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:14.227826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d9faa-36dd-4c9d-8c5d-789e6c0029e3 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:14.227826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d9faa-36dd-4c9d-8c5d-789e6c0029e3 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:14.229499 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 751/1485] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:27:14 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:27:14.235870 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c409e6f-c2f3-42a7-a1b7-b6b5f2c0099d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:14.236458 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c409e6f-c2f3-42a7-a1b7-b6b5f2c0099d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:14.236458 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c409e6f-c2f3-42a7-a1b7-b6b5f2c0099d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:14.309896 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8c409e6f-c2f3-42a7-a1b7-b6b5f2c0099d None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:27:14.311148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8c409e6f-c2f3-42a7-a1b7-b6b5f2c0099d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:14.412475 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 735/1486] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:27:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:27:14.452095 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d9faa-36dd-4c9d-8c5d-789e6c0029e3 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:14.453705 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d44d9faa-36dd-4c9d-8c5d-789e6c0029e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:14.569435 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d9faa-36dd-4c9d-8c5d-789e6c0029e3 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:14.653716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:27:14.655889 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba5fb7-5051-4947-8ec1-6ebacd686d1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:14.656727 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba5fb7-5051-4947-8ec1-6ebacd686d1a service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:14.657459 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba5fb7-5051-4947-8ec1-6ebacd686d1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:14.658110 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba5fb7-5051-4947-8ec1-6ebacd686d1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:14.660312 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-22ba5fb7-5051-4947-8ec1-6ebacd686d1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:14.781023 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba5fb7-5051-4947-8ec1-6ebacd686d1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:14.782913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba5fb7-5051-4947-8ec1-6ebacd686d1a service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:14.803226 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 752/1487] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:27:14 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:27:17.005019 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67efce5-7fcb-41d9-bceb-680877bf1a6b None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:17.005757 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67efce5-7fcb-41d9-bceb-680877bf1a6b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:17.005757 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67efce5-7fcb-41d9-bceb-680877bf1a6b None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:17.007039 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 736/1488] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:27:17 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:27:17.014715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677981d-fc65-40a2-9043-f0cc0243ba6c None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:17.015245 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677981d-fc65-40a2-9043-f0cc0243ba6c None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:17.015406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677981d-fc65-40a2-9043-f0cc0243ba6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:17.096177 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-5677981d-fc65-40a2-9043-f0cc0243ba6c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:27:17.097426 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-5677981d-fc65-40a2-9043-f0cc0243ba6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:17.158679 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:27:17.204966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 753/1489] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:27:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:27:17.255354 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67efce5-7fcb-41d9-bceb-680877bf1a6b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:17.256911 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e67efce5-7fcb-41d9-bceb-680877bf1a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:17.300275 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:27:17.379090 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67efce5-7fcb-41d9-bceb-680877bf1a6b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:17.457476 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2f868-8dab-48c5-97c8-b5132892de6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:17.458113 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2f868-8dab-48c5-97c8-b5132892de6e service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:17.460404 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2f868-8dab-48c5-97c8-b5132892de6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:17.460404 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2f868-8dab-48c5-97c8-b5132892de6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:17.460404 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-59a2f868-8dab-48c5-97c8-b5132892de6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:17.557151 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:27:17.590456 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2f868-8dab-48c5-97c8-b5132892de6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:17.591983 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2f868-8dab-48c5-97c8-b5132892de6e service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:17.611584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 737/1490] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:27:17 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:27:19.896818 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b8baf-c12a-4c97-9a76-734112d729c9 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:19.897478 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b8baf-c12a-4c97-9a76-734112d729c9 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:19.897478 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b8baf-c12a-4c97-9a76-734112d729c9 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:19.898644 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 754/1491] 10.222.0.121 () {54 vars in 932 bytes} [Thu Aug 4 10:27:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:27:19.905234 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be5f18-da9d-4620-8738-6e470e241395 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:19.905908 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be5f18-da9d-4620-8738-6e470e241395 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:19.905908 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be5f18-da9d-4620-8738-6e470e241395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:19.985844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-93be5f18-da9d-4620-8738-6e470e241395 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:27:19.987124 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-93be5f18-da9d-4620-8738-6e470e241395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:20.098835 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 738/1492] 10.222.0.121 () {58 vars in 1029 bytes} [Thu Aug 4 10:27:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:27:20.144094 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b8baf-c12a-4c97-9a76-734112d729c9 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:27:20.146023 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-1d1b8baf-c12a-4c97-9a76-734112d729c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:20.262360 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b8baf-c12a-4c97-9a76-734112d729c9 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:27:20.342519 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7957c73-dd34-4f26-9567-fb709ee3fa95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:27:20.343167 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7957c73-dd34-4f26-9567-fb709ee3fa95 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:27:20.343455 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7957c73-dd34-4f26-9567-fb709ee3fa95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:27:20.343759 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7957c73-dd34-4f26-9567-fb709ee3fa95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:27:20.345528 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c7957c73-dd34-4f26-9567-fb709ee3fa95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:27:20.356062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:27:20.470181 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7957c73-dd34-4f26-9567-fb709ee3fa95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:27:20.471708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7957c73-dd34-4f26-9567-fb709ee3fa95 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:27:20.492295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 755/1493] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:27:20 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:27:30.096655 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=82 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:27:30.489467 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=50 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:28:59.181128 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c207b32-41bd-4917-92e1-dbc120015c68 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:59.181905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c207b32-41bd-4917-92e1-dbc120015c68 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:59.182140 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c207b32-41bd-4917-92e1-dbc120015c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:59.292110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9c207b32-41bd-4917-92e1-dbc120015c68 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:28:59.293898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9c207b32-41bd-4917-92e1-dbc120015c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:59.449296 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 739/1494] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:28:59 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:28:59.458331 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe6cbb-4d83-458a-a90e-2741d3d39bc1 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:59.459008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe6cbb-4d83-458a-a90e-2741d3d39bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:59.459008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe6cbb-4d83-458a-a90e-2741d3d39bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:59.550967 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-4ebe6cbb-4d83-458a-a90e-2741d3d39bc1 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:28:59.552313 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-4ebe6cbb-4d83-458a-a90e-2741d3d39bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:59.688384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 756/1495] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:28:59 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:28:59.693571 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c207b32-41bd-4917-92e1-dbc120015c68 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:28:59.694880 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9c207b32-41bd-4917-92e1-dbc120015c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:28:59.716028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c207b32-41bd-4917-92e1-dbc120015c68 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:28:59.818217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:28:59.885159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb3fc2a-29b7-4726-aca4-791ab170224a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:28:59.885839 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3fc2a-29b7-4726-aca4-791ab170224a admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:28:59.886119 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3fc2a-29b7-4726-aca4-791ab170224a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:28:59.886394 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3fc2a-29b7-4726-aca4-791ab170224a admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:28:59.887115 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb3fc2a-29b7-4726-aca4-791ab170224a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:28:59.887425 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb3fc2a-29b7-4726-aca4-791ab170224a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:28:59.888493 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: WARNING py.warnings [None req-acb3fc2a-29b7-4726-aca4-791ab170224a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:28:59.888493 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: warnings.warn(msg) Aug 04 10:28:59.888493 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: Aug 04 10:28:59.889410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb3fc2a-29b7-4726-aca4-791ab170224a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:28:59.904382 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 740/1496] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:28:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:28:59.914039 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe6cbb-4d83-458a-a90e-2741d3d39bc1 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:28:59.925002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe6cbb-4d83-458a-a90e-2741d3d39bc1 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:28:59.998862 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:29:00.043049 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c9cb3-ee77-4977-b1c3-9bef64c7cb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:29:00.043694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c9cb3-ee77-4977-b1c3-9bef64c7cb99 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:29:00.044012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c9cb3-ee77-4977-b1c3-9bef64c7cb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:29:00.044284 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c9cb3-ee77-4977-b1c3-9bef64c7cb99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:29:00.044987 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c9cb3-ee77-4977-b1c3-9bef64c7cb99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:29:00.046834 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: WARNING py.warnings [None req-278c9cb3-ee77-4977-b1c3-9bef64c7cb99 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:29:00.046834 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: warnings.warn(msg) Aug 04 10:29:00.046834 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: Aug 04 10:29:00.048078 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c9cb3-ee77-4977-b1c3-9bef64c7cb99 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:29:00.115696 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 757/1497] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:28:59 2022] POST /identity/v3/projects => generated 380 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:29:00.121270 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb3fc2a-29b7-4726-aca4-791ab170224a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:29:00.123060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-acb3fc2a-29b7-4726-aca4-791ab170224a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:29:00.170994 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb3fc2a-29b7-4726-aca4-791ab170224a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:29:00.290644 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1665a45-90a8-44dc-9399-0f0f0faf3f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:29:00.291526 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1665a45-90a8-44dc-9399-0f0f0faf3f74 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:29:00.291526 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1665a45-90a8-44dc-9399-0f0f0faf3f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:29:00.291898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1665a45-90a8-44dc-9399-0f0f0faf3f74 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:29:00.292710 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1665a45-90a8-44dc-9399-0f0f0faf3f74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:29:00.294079 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: WARNING py.warnings [None req-b1665a45-90a8-44dc-9399-0f0f0faf3f74 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:29:00.294079 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: warnings.warn(msg) Aug 04 10:29:00.294079 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: Aug 04 10:29:00.295530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1665a45-90a8-44dc-9399-0f0f0faf3f74 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:29:00.348581 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 741/1498] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:29:00 2022] POST /identity/v3/users => generated 291 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:29:00.355477 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c9cb3-ee77-4977-b1c3-9bef64c7cb99 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:29:00.358685 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-278c9cb3-ee77-4977-b1c3-9bef64c7cb99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:29:00.480378 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c9cb3-ee77-4977-b1c3-9bef64c7cb99 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:29:00.617292 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:29:00.635697 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146072e-3b67-4574-b016-1f75f469ddf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:29:00.636445 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146072e-3b67-4574-b016-1f75f469ddf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:29:00.636792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146072e-3b67-4574-b016-1f75f469ddf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:29:00.637114 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146072e-3b67-4574-b016-1f75f469ddf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:29:00.637894 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146072e-3b67-4574-b016-1f75f469ddf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:29:00.639008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: WARNING py.warnings [None req-b146072e-3b67-4574-b016-1f75f469ddf2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:29:00.639008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: warnings.warn(msg) Aug 04 10:29:00.639008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: Aug 04 10:29:00.640144 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146072e-3b67-4574-b016-1f75f469ddf2 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:29:00.655982 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 758/1499] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:29:00 2022] GET /identity/v3/roles => generated 1427 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:29:00.661029 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1665a45-90a8-44dc-9399-0f0f0faf3f74 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:29:00.663089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b1665a45-90a8-44dc-9399-0f0f0faf3f74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:29:00.676009 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1665a45-90a8-44dc-9399-0f0f0faf3f74 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:29:00.706339 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:29:00.803851 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366a25b-3fbc-426d-b4d4-1bc11113ca0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:29:00.804787 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366a25b-3fbc-426d-b4d4-1bc11113ca0c admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:29:00.805074 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366a25b-3fbc-426d-b4d4-1bc11113ca0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:29:00.805336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366a25b-3fbc-426d-b4d4-1bc11113ca0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:29:00.806093 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a366a25b-3fbc-426d-b4d4-1bc11113ca0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:29:00.807243 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: WARNING py.warnings [None req-a366a25b-3fbc-426d-b4d4-1bc11113ca0c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:29:00.807243 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: warnings.warn(msg) Aug 04 10:29:00.807243 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: Aug 04 10:29:00.808313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a366a25b-3fbc-426d-b4d4-1bc11113ca0c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:29:00.824925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 742/1500] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:29:00 2022] GET /identity/v3/roles => generated 1427 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:29:00.830081 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146072e-3b67-4574-b016-1f75f469ddf2 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:29:00.831833 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-b146072e-3b67-4574-b016-1f75f469ddf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:29:00.876079 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146072e-3b67-4574-b016-1f75f469ddf2 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:29:00.997182 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf4960f-2dc6-493d-9ea9-a927643e90e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:29:00.997977 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4960f-2dc6-493d-9ea9-a927643e90e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:29:00.998269 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4960f-2dc6-493d-9ea9-a927643e90e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:29:00.998549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4960f-2dc6-493d-9ea9-a927643e90e0 admin admin] PATH_INFO: `/v3/projects/867716dd8cb948298050e25778ef4f57/users/6ecf231918ee48d2b51f32c56ac6e531/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:29:01.027884 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf4960f-2dc6-493d-9ea9-a927643e90e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=867716dd8cb948298050e25778ef4f57, user_id=6ecf231918ee48d2b51f32c56ac6e531, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:29:01.029114 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: WARNING py.warnings [None req-5bf4960f-2dc6-493d-9ea9-a927643e90e0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:29:01.029114 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: warnings.warn(msg) Aug 04 10:29:01.029114 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: Aug 04 10:29:01.031404 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf4960f-2dc6-493d-9ea9-a927643e90e0 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:29:01.063665 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 759/1501] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:29:00 2022] PUT /identity/v3/projects/867716dd8cb948298050e25778ef4f57/users/6ecf231918ee48d2b51f32c56ac6e531/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:29:01.077430 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651de6e4-1c0d-4b2a-a31d-5397b1bcc76f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:29:01.077792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651de6e4-1c0d-4b2a-a31d-5397b1bcc76f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:29:01.078002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651de6e4-1c0d-4b2a-a31d-5397b1bcc76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:29:01.149197 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-651de6e4-1c0d-4b2a-a31d-5397b1bcc76f None None] MFA Rules not processed for user `6ecf231918ee48d2b51f32c56ac6e531`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:29:01.150393 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-651de6e4-1c0d-4b2a-a31d-5397b1bcc76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:29:01.247482 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 743/1502] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:29:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:29:01.261453 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003d6be-bd2c-44da-ba56-9deb7c92cc3a None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:29:01.261770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003d6be-bd2c-44da-ba56-9deb7c92cc3a None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:29:01.262018 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003d6be-bd2c-44da-ba56-9deb7c92cc3a None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:29:01.263564 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 760/1503] 10.222.0.121 () {54 vars in 969 bytes} [Thu Aug 4 10:29:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:29:01.269555 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34773a15-7233-4f31-a2b6-1d507c7f4671 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:29:01.269852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34773a15-7233-4f31-a2b6-1d507c7f4671 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:29:01.270155 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34773a15-7233-4f31-a2b6-1d507c7f4671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:29:01.326492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:29:01.383292 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-34773a15-7233-4f31-a2b6-1d507c7f4671 None None] MFA Rules not processed for user `c19bb76d0e2544299916007fbfa62578`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:29:01.384616 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-34773a15-7233-4f31-a2b6-1d507c7f4671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:29:01.475430 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 744/1504] 10.222.0.121 () {58 vars in 1066 bytes} [Thu Aug 4 10:29:01 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:29:01.489401 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003d6be-bd2c-44da-ba56-9deb7c92cc3a None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:29:01.491466 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-3003d6be-bd2c-44da-ba56-9deb7c92cc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:29:01.555574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:29:01.643463 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003d6be-bd2c-44da-ba56-9deb7c92cc3a None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:29:01.732478 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1c346-cf73-4e7a-b22e-959059a6ff4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:29:01.732877 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1c346-cf73-4e7a-b22e-959059a6ff4a service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:29:01.733163 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1c346-cf73-4e7a-b22e-959059a6ff4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:29:01.733443 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1c346-cf73-4e7a-b22e-959059a6ff4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:29:01.735187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-cfb1c346-cf73-4e7a-b22e-959059a6ff4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:29:01.859599 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb1c346-cf73-4e7a-b22e-959059a6ff4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:29:01.861131 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb1c346-cf73-4e7a-b22e-959059a6ff4a service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:29:01.865698 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:29:01.890576 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 761/1505] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:29:01 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:29:01.947867 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875088a-c2ee-4c9e-b9b9-0eff3bf92daa None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:29:01.948216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875088a-c2ee-4c9e-b9b9-0eff3bf92daa None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:29:01.948471 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875088a-c2ee-4c9e-b9b9-0eff3bf92daa None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:29:01.949892 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 745/1506] 10.222.0.121 () {52 vars in 835 bytes} [Thu Aug 4 10:29:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:29:01.954562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1c346-cf73-4e7a-b22e-959059a6ff4a service cinder] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:29:01.955990 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-cfb1c346-cf73-4e7a-b22e-959059a6ff4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:29:02.082140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1c346-cf73-4e7a-b22e-959059a6ff4a service cinder] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:29:02.216027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce6d0c4-90a3-4d3a-bf2b-ceccf4b3bdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:29:02.216710 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6d0c4-90a3-4d3a-bf2b-ceccf4b3bdb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:29:02.216978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6d0c4-90a3-4d3a-bf2b-ceccf4b3bdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:29:02.217260 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6d0c4-90a3-4d3a-bf2b-ceccf4b3bdb9 admin admin] PATH_INFO: `/v3/users/6ecf231918ee48d2b51f32c56ac6e531` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:29:02.236769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce6d0c4-90a3-4d3a-bf2b-ceccf4b3bdb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ecf231918ee48d2b51f32c56ac6e531)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:29:02.237953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: WARNING py.warnings [None req-6ce6d0c4-90a3-4d3a-bf2b-ceccf4b3bdb9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:29:02.237953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: warnings.warn(msg) Aug 04 10:29:02.237953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: Aug 04 10:29:02.239458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce6d0c4-90a3-4d3a-bf2b-ceccf4b3bdb9 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:29:02.253857 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:29:02.332334 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6ce6d0c4-90a3-4d3a-bf2b-ceccf4b3bdb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ecf231918ee48d2b51f32c56ac6e531'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:29:02.332875 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6ce6d0c4-90a3-4d3a-bf2b-ceccf4b3bdb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ecf231918ee48d2b51f32c56ac6e531'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:29:02.364422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6ce6d0c4-90a3-4d3a-bf2b-ceccf4b3bdb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ecf231918ee48d2b51f32c56ac6e531'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:29:02.383197 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6ce6d0c4-90a3-4d3a-bf2b-ceccf4b3bdb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ecf231918ee48d2b51f32c56ac6e531'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:29:02.395703 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6ce6d0c4-90a3-4d3a-bf2b-ceccf4b3bdb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ecf231918ee48d2b51f32c56ac6e531'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:29:02.403402 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 762/1507] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:29:01 2022] DELETE /identity/v3/users/6ecf231918ee48d2b51f32c56ac6e531 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:29:02.410161 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020bf45-5b12-43d2-87cf-debc37932f14 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:29:02.412430 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020bf45-5b12-43d2-87cf-debc37932f14 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:29:02.412430 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020bf45-5b12-43d2-87cf-debc37932f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:29:02.506983 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-1020bf45-5b12-43d2-87cf-debc37932f14 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:29:02.508257 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1020bf45-5b12-43d2-87cf-debc37932f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:29:02.609434 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:29:02.628392 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 746/1508] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:29:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:29:02.639805 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce6d0c4-90a3-4d3a-bf2b-ceccf4b3bdb9 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:29:02.641116 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6ce6d0c4-90a3-4d3a-bf2b-ceccf4b3bdb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:29:02.768928 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce6d0c4-90a3-4d3a-bf2b-ceccf4b3bdb9 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:29:02.855435 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:29:02.884735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617521ab-dffe-4885-bb80-5d517f3ae90f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:29:02.885395 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617521ab-dffe-4885-bb80-5d517f3ae90f service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:29:02.885678 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617521ab-dffe-4885-bb80-5d517f3ae90f service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:29:02.885952 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617521ab-dffe-4885-bb80-5d517f3ae90f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:29:02.887714 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-617521ab-dffe-4885-bb80-5d517f3ae90f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:29:03.055895 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617521ab-dffe-4885-bb80-5d517f3ae90f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:29:03.057369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617521ab-dffe-4885-bb80-5d517f3ae90f service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:29:03.080886 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 763/1509] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:29:02 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:29:03.535297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020bf45-5b12-43d2-87cf-debc37932f14 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:29:03.536751 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1020bf45-5b12-43d2-87cf-debc37932f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:29:03.550770 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020bf45-5b12-43d2-87cf-debc37932f14 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:29:03.688799 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699cad5e-a28c-4f66-a273-c865c0f68833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:29:03.689455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cad5e-a28c-4f66-a273-c865c0f68833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:29:03.689864 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cad5e-a28c-4f66-a273-c865c0f68833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:29:03.690011 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cad5e-a28c-4f66-a273-c865c0f68833 admin admin] PATH_INFO: `/v3/projects/867716dd8cb948298050e25778ef4f57` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:29:03.702771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699cad5e-a28c-4f66-a273-c865c0f68833 admin admin] RBAC: Authorizing `identity:delete_project(project_id=867716dd8cb948298050e25778ef4f57)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:29:03.703852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: WARNING py.warnings [None req-699cad5e-a28c-4f66-a273-c865c0f68833 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:29:03.703852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: warnings.warn(msg) Aug 04 10:29:03.703852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: Aug 04 10:29:03.705202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699cad5e-a28c-4f66-a273-c865c0f68833 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:29:03.798744 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-699cad5e-a28c-4f66-a273-c865c0f68833 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '867716dd8cb948298050e25778ef4f57', 'request_id': 'req-699cad5e-a28c-4f66-a273-c865c0f68833'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:29:03.818031 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-699cad5e-a28c-4f66-a273-c865c0f68833 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '867716dd8cb948298050e25778ef4f57', 'request_id': 'req-699cad5e-a28c-4f66-a273-c865c0f68833'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:29:03.827496 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-699cad5e-a28c-4f66-a273-c865c0f68833 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '867716dd8cb948298050e25778ef4f57', 'request_id': 'req-699cad5e-a28c-4f66-a273-c865c0f68833'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:29:03.863273 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-699cad5e-a28c-4f66-a273-c865c0f68833 admin admin] The token cache is being invalidate because project 867716dd8cb948298050e25778ef4f57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:29:03.863864 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-699cad5e-a28c-4f66-a273-c865c0f68833 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:29:03.864507 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-699cad5e-a28c-4f66-a273-c865c0f68833 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:29:03.866185 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 747/1510] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:29:03 2022] DELETE /identity/v3/projects/867716dd8cb948298050e25778ef4f57 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:29:13.078174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=62 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:29:13.860362 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:30:40.461369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580fb67-ae8d-4af1-bfa9-fe9a644e5154 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:40.462287 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580fb67-ae8d-4af1-bfa9-fe9a644e5154 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:40.462287 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580fb67-ae8d-4af1-bfa9-fe9a644e5154 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:40.463327 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 10:30:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:30:40.469356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19e2c7-6bfe-45bb-b844-c584289f8da3 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:40.469955 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19e2c7-6bfe-45bb-b844-c584289f8da3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:40.469955 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19e2c7-6bfe-45bb-b844-c584289f8da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:40.582971 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-fe19e2c7-6bfe-45bb-b844-c584289f8da3 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:30:40.584523 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fe19e2c7-6bfe-45bb-b844-c584289f8da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:40.706403 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 10:30:40 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:30:40.863872 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580fb67-ae8d-4af1-bfa9-fe9a644e5154 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:30:40.866237 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c580fb67-ae8d-4af1-bfa9-fe9a644e5154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:41.002962 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580fb67-ae8d-4af1-bfa9-fe9a644e5154 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:30:41.009132 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840778b-24f2-4ee3-a716-73f62c5ffbb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:30:41.009804 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840778b-24f2-4ee3-a716-73f62c5ffbb0 service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:41.010088 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840778b-24f2-4ee3-a716-73f62c5ffbb0 service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:41.010370 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840778b-24f2-4ee3-a716-73f62c5ffbb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:41.019531 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840778b-24f2-4ee3-a716-73f62c5ffbb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:30:41.021207 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840778b-24f2-4ee3-a716-73f62c5ffbb0 service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:30:41.031858 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 765/1513] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:30:40 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:30:41.913082 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6c531-359a-4b1e-9810-100023a8b52b None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:41.913746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6c531-359a-4b1e-9810-100023a8b52b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:41.913746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6c531-359a-4b1e-9810-100023a8b52b None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:41.915326 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 749/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 10:30:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:30:41.920563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20f418-40da-4c97-846b-bc7a50442ca3 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:41.921141 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20f418-40da-4c97-846b-bc7a50442ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:41.921141 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20f418-40da-4c97-846b-bc7a50442ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:41.944296 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-da20f418-40da-4c97-846b-bc7a50442ca3 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:30:41.945540 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-da20f418-40da-4c97-846b-bc7a50442ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:41.954376 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 766/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 10:30:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:30:42.106775 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee6c531-359a-4b1e-9810-100023a8b52b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:30:42.122505 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee6c531-359a-4b1e-9810-100023a8b52b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:30:42.129216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a18088-7a26-42d8-9b8f-e61802dacedd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:30:42.129875 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a18088-7a26-42d8-9b8f-e61802dacedd service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:42.130160 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a18088-7a26-42d8-9b8f-e61802dacedd service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:42.130440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a18088-7a26-42d8-9b8f-e61802dacedd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:42.139915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a18088-7a26-42d8-9b8f-e61802dacedd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:30:42.141872 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a18088-7a26-42d8-9b8f-e61802dacedd service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:30:42.152934 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 750/1516] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:30:42 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:30:42.463166 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b99ee3-5bd6-427a-a933-bdd0ccb42d3f None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:42.463580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b99ee3-5bd6-427a-a933-bdd0ccb42d3f None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:42.464276 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b99ee3-5bd6-427a-a933-bdd0ccb42d3f None None] PATH_INFO: `/v3` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:42.465888 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 767/1517] 10.222.0.107 () {54 vars in 930 bytes} [Thu Aug 4 10:30:42 2022] GET /identity/v3 => generated 256 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:30:42.473268 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d8d53-e069-45da-8a07-d9262f97dc38 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:42.473700 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d8d53-e069-45da-8a07-d9262f97dc38 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:42.474102 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d8d53-e069-45da-8a07-d9262f97dc38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:42.586856 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ea0d8d53-e069-45da-8a07-d9262f97dc38 None None] MFA Rules not processed for user `01c7cfbe911c4378857848966f4f8d98`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:30:42.588258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ea0d8d53-e069-45da-8a07-d9262f97dc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:42.617359 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:30:42.688262 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 751/1518] 10.222.0.107 () {58 vars in 1020 bytes} [Thu Aug 4 10:30:42 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:30:51.938819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=56 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:30:52.609473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4254c-7aa2-42bd-a824-67724c57c72c None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:52.610051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4254c-7aa2-42bd-a824-67724c57c72c None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:52.610475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4254c-7aa2-42bd-a824-67724c57c72c None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:52.612127 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 768/1519] 10.222.0.121 () {54 vars in 966 bytes} [Thu Aug 4 10:30:52 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:30:52.618642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ad76e-42f7-487e-806e-81392e48f449 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:52.619293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ad76e-42f7-487e-806e-81392e48f449 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:52.619293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ad76e-42f7-487e-806e-81392e48f449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:52.642762 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-048ad76e-42f7-487e-806e-81392e48f449 None None] MFA Rules not processed for user `01c7cfbe911c4378857848966f4f8d98`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:30:52.644244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-048ad76e-42f7-487e-806e-81392e48f449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:52.655114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 752/1520] 10.222.0.121 () {58 vars in 1063 bytes} [Thu Aug 4 10:30:52 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:30:52.663325 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4254c-7aa2-42bd-a824-67724c57c72c None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:30:52.678468 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4254c-7aa2-42bd-a824-67724c57c72c None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:30:52.683808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc189b6-7401-4d52-b654-98fc906450e9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c7cfbe911c4378857848966f4f8d98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:30:52.684505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc189b6-7401-4d52-b654-98fc906450e9 service placement] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:52.684792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc189b6-7401-4d52-b654-98fc906450e9 service placement] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:52.685078 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc189b6-7401-4d52-b654-98fc906450e9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:52.695417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc189b6-7401-4d52-b654-98fc906450e9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:30:52.697341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc189b6-7401-4d52-b654-98fc906450e9 service placement] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:30:52.711443 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 769/1521] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:30:52 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:30:53.615808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ec63f-0d15-46d5-a6a7-c0e937792a50 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:53.616346 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ec63f-0d15-46d5-a6a7-c0e937792a50 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:53.616784 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ec63f-0d15-46d5-a6a7-c0e937792a50 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:53.618680 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 753/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 10:30:53 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:30:53.624284 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787affe1-0bcd-44ea-8905-80c4ceec75fd None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:53.624863 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787affe1-0bcd-44ea-8905-80c4ceec75fd None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:53.624863 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787affe1-0bcd-44ea-8905-80c4ceec75fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:53.653248 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-787affe1-0bcd-44ea-8905-80c4ceec75fd None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:30:53.654450 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-787affe1-0bcd-44ea-8905-80c4ceec75fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:53.662853 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 770/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 10:30:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:30:53.832926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ec63f-0d15-46d5-a6a7-c0e937792a50 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:30:53.850658 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ec63f-0d15-46d5-a6a7-c0e937792a50 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:30:53.858333 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ee631-a6fa-404b-82c4-ef55ff73a72b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:30:53.859048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ee631-a6fa-404b-82c4-ef55ff73a72b service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:53.859535 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ee631-a6fa-404b-82c4-ef55ff73a72b service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:53.859723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ee631-a6fa-404b-82c4-ef55ff73a72b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:53.870797 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ee631-a6fa-404b-82c4-ef55ff73a72b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:30:53.872624 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ee631-a6fa-404b-82c4-ef55ff73a72b service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:30:53.884620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 754/1524] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:30:53 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:30:54.800809 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4403d-047f-475a-809b-2588b2249a84 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:54.801455 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4403d-047f-475a-809b-2588b2249a84 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:54.801455 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4403d-047f-475a-809b-2588b2249a84 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:54.803552 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 771/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 10:30:54 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:30:54.809845 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1591a-f1e0-4c61-b82b-facf1bfd784a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:54.810413 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1591a-f1e0-4c61-b82b-facf1bfd784a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:54.810413 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1591a-f1e0-4c61-b82b-facf1bfd784a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:54.832857 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-1db1591a-f1e0-4c61-b82b-facf1bfd784a None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:30:54.834126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1db1591a-f1e0-4c61-b82b-facf1bfd784a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:54.844500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 755/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 10:30:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:30:55.008289 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa4403d-047f-475a-809b-2588b2249a84 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:30:55.027411 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa4403d-047f-475a-809b-2588b2249a84 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:30:55.034600 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4a015-3b6f-418f-b7d0-3aa11166429a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:30:55.035248 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4a015-3b6f-418f-b7d0-3aa11166429a service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:55.035621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4a015-3b6f-418f-b7d0-3aa11166429a service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:55.035918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4a015-3b6f-418f-b7d0-3aa11166429a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:55.046020 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4a015-3b6f-418f-b7d0-3aa11166429a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:30:55.047873 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4a015-3b6f-418f-b7d0-3aa11166429a service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:30:55.059014 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 772/1527] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:30:55 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:30:59.097155 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd82fa-abdd-4b54-a6a3-486f1582fc11 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:59.097966 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd82fa-abdd-4b54-a6a3-486f1582fc11 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:59.097966 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd82fa-abdd-4b54-a6a3-486f1582fc11 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:59.099106 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 756/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:30:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:30:59.105957 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1513e-c17f-4662-88e1-8fb843d95aa4 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:59.106613 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1513e-c17f-4662-88e1-8fb843d95aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:59.106613 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1513e-c17f-4662-88e1-8fb843d95aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:59.131502 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-9ac1513e-c17f-4662-88e1-8fb843d95aa4 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:30:59.133202 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-9ac1513e-c17f-4662-88e1-8fb843d95aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:59.141656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 773/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:30:59 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:30:59.226330 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd82fa-abdd-4b54-a6a3-486f1582fc11 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:30:59.228578 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a4bd82fa-abdd-4b54-a6a3-486f1582fc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:30:59.325147 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd82fa-abdd-4b54-a6a3-486f1582fc11 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:30:59.329032 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f6c1e-b173-4a7f-925d-848d47a2a06a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:30:59.329678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f6c1e-b173-4a7f-925d-848d47a2a06a service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:30:59.329959 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f6c1e-b173-4a7f-925d-848d47a2a06a service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:30:59.330245 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f6c1e-b173-4a7f-925d-848d47a2a06a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:30:59.340075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f6c1e-b173-4a7f-925d-848d47a2a06a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:30:59.341557 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f6c1e-b173-4a7f-925d-848d47a2a06a service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:30:59.353393 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 757/1530] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:30:59 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:31:01.575790 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56c891-8f17-4b24-8481-da55f5d59582 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:01.576647 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56c891-8f17-4b24-8481-da55f5d59582 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:01.576647 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56c891-8f17-4b24-8481-da55f5d59582 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:01.578284 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 774/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:31:01 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:31:01.586276 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a1cd2-829c-4456-a303-76505dcc8f15 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:01.586770 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a1cd2-829c-4456-a303-76505dcc8f15 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:01.587310 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a1cd2-829c-4456-a303-76505dcc8f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:01.615304 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4f9a1cd2-829c-4456-a303-76505dcc8f15 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:01.616629 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4f9a1cd2-829c-4456-a303-76505dcc8f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:01.625421 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 758/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:31:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:31:01.666204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56c891-8f17-4b24-8481-da55f5d59582 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:01.668400 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-de56c891-8f17-4b24-8481-da55f5d59582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:01.684590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56c891-8f17-4b24-8481-da55f5d59582 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:01.688807 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eab0bc-09c7-45c5-afa5-dce9de297524 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:01.689552 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eab0bc-09c7-45c5-afa5-dce9de297524 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:01.689842 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eab0bc-09c7-45c5-afa5-dce9de297524 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:01.690125 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eab0bc-09c7-45c5-afa5-dce9de297524 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:01.701190 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eab0bc-09c7-45c5-afa5-dce9de297524 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:01.702792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eab0bc-09c7-45c5-afa5-dce9de297524 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:01.715482 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 775/1533] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:31:01 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:31:05.518712 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28252f7-8e48-4541-bbc2-5c4df15b9101 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:05.519385 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28252f7-8e48-4541-bbc2-5c4df15b9101 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:05.519385 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28252f7-8e48-4541-bbc2-5c4df15b9101 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:05.520884 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 759/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:31:05 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:31:05.527890 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a472ac-a9a1-44d2-8961-c8bc39764a7c None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:05.529206 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a472ac-a9a1-44d2-8961-c8bc39764a7c None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:05.529206 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a472ac-a9a1-44d2-8961-c8bc39764a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:05.554893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-27a472ac-a9a1-44d2-8961-c8bc39764a7c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:05.556205 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-27a472ac-a9a1-44d2-8961-c8bc39764a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:05.568311 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 776/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:31:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:31:05.883638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28252f7-8e48-4541-bbc2-5c4df15b9101 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:05.885625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f28252f7-8e48-4541-bbc2-5c4df15b9101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:05.972440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:05.991900 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28252f7-8e48-4541-bbc2-5c4df15b9101 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:05.996631 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d20770-9a5c-465a-8ece-5bf7283bb9e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:05.997352 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d20770-9a5c-465a-8ece-5bf7283bb9e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:05.997691 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d20770-9a5c-465a-8ece-5bf7283bb9e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:05.998018 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d20770-9a5c-465a-8ece-5bf7283bb9e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:06.009579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d20770-9a5c-465a-8ece-5bf7283bb9e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:06.011501 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d20770-9a5c-465a-8ece-5bf7283bb9e7 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:06.025220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 760/1536] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:31:05 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:31:08.229325 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241a1db-aa3d-4e1f-8006-45ca066297ba None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:08.230053 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241a1db-aa3d-4e1f-8006-45ca066297ba None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:08.230053 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241a1db-aa3d-4e1f-8006-45ca066297ba None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:08.231464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 777/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 10:31:08 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:31:08.237280 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2923795-2164-485e-a2a5-f7a24a82bf93 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:08.237851 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2923795-2164-485e-a2a5-f7a24a82bf93 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:08.237851 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2923795-2164-485e-a2a5-f7a24a82bf93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:08.258600 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a2923795-2164-485e-a2a5-f7a24a82bf93 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:31:08.259859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a2923795-2164-485e-a2a5-f7a24a82bf93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:08.269502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 761/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 10:31:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:31:08.579321 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241a1db-aa3d-4e1f-8006-45ca066297ba None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:31:08.581377 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-3241a1db-aa3d-4e1f-8006-45ca066297ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:31:08.597974 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241a1db-aa3d-4e1f-8006-45ca066297ba None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:31:08.602513 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c750759-b7b3-455a-8e01-80ae673823ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:31:08.603179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c750759-b7b3-455a-8e01-80ae673823ad service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:31:08.603593 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c750759-b7b3-455a-8e01-80ae673823ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:31:08.603876 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c750759-b7b3-455a-8e01-80ae673823ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:31:08.616625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c750759-b7b3-455a-8e01-80ae673823ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:31:08.618147 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c750759-b7b3-455a-8e01-80ae673823ad service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:31:08.629947 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 778/1539] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:31:08 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:31:18.253603 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=11 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:31:18.624805 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=33 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:35.741617 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d678be-085c-4bbf-960b-b8fb3d952667 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:35.743714 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d678be-085c-4bbf-960b-b8fb3d952667 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:35.743714 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d678be-085c-4bbf-960b-b8fb3d952667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:35.782222 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-84d678be-085c-4bbf-960b-b8fb3d952667 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:35.783695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-84d678be-085c-4bbf-960b-b8fb3d952667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:35.806171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 762/1540] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:35.817134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7003a0-4d66-4348-b4cf-a0ce64e899a1 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:35.817944 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7003a0-4d66-4348-b4cf-a0ce64e899a1 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:35.817944 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7003a0-4d66-4348-b4cf-a0ce64e899a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:35.855424 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-de7003a0-4d66-4348-b4cf-a0ce64e899a1 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:35.856959 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-de7003a0-4d66-4348-b4cf-a0ce64e899a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:35.872903 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 779/1541] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:35.878671 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d678be-085c-4bbf-960b-b8fb3d952667 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:35.896663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d678be-085c-4bbf-960b-b8fb3d952667 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:35.909309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca0489-e541-4715-bdb6-2b473cd4ffde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:35.910302 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca0489-e541-4715-bdb6-2b473cd4ffde admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:35.910797 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca0489-e541-4715-bdb6-2b473cd4ffde admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:35.911275 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca0489-e541-4715-bdb6-2b473cd4ffde admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:35.912606 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca0489-e541-4715-bdb6-2b473cd4ffde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:32:35.913151 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca0489-e541-4715-bdb6-2b473cd4ffde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:35.915531 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca0489-e541-4715-bdb6-2b473cd4ffde admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:35.939326 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea4d3d-76d0-45b0-a7d7-f76ca863e4f0 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:35.939613 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea4d3d-76d0-45b0-a7d7-f76ca863e4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:35.939867 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea4d3d-76d0-45b0-a7d7-f76ca863e4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:35.948746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 763/1542] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:32:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:32:35.961962 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca0489-e541-4715-bdb6-2b473cd4ffde admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:35.975451 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-bfea4d3d-76d0-45b0-a7d7-f76ca863e4f0 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:35.976790 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-bfea4d3d-76d0-45b0-a7d7-f76ca863e4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:35.980542 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca0489-e541-4715-bdb6-2b473cd4ffde admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:35.989791 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead8b16-cc14-4fca-a493-96eb2b2cc352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:35.990455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead8b16-cc14-4fca-a493-96eb2b2cc352 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:35.990743 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead8b16-cc14-4fca-a493-96eb2b2cc352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:35.991020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead8b16-cc14-4fca-a493-96eb2b2cc352 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:35.991846 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead8b16-cc14-4fca-a493-96eb2b2cc352 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:35.993015 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: WARNING py.warnings [None req-5ead8b16-cc14-4fca-a493-96eb2b2cc352 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:32:35.993015 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: warnings.warn(msg) Aug 04 10:32:35.993015 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: Aug 04 10:32:35.994367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead8b16-cc14-4fca-a493-96eb2b2cc352 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:36.002911 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 780/1543] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:36.010139 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa23050-3e7b-4d6a-be36-2d6869c45276 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.010504 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa23050-3e7b-4d6a-be36-2d6869c45276 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.010722 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa23050-3e7b-4d6a-be36-2d6869c45276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.065088 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-caa23050-3e7b-4d6a-be36-2d6869c45276 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:36.066572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-caa23050-3e7b-4d6a-be36-2d6869c45276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:36.086502 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 781/1544] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:32:36.088061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb23fdf-98ab-40e7-9974-5df2d7e93ea2 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.088359 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb23fdf-98ab-40e7-9974-5df2d7e93ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.088878 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb23fdf-98ab-40e7-9974-5df2d7e93ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.105054 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 764/1545] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:32:35 2022] POST /identity/v3/projects => generated 376 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:32:36.107122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29dea0-8786-4065-a0de-19e2451e2a51 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.107398 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29dea0-8786-4065-a0de-19e2451e2a51 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.111989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29dea0-8786-4065-a0de-19e2451e2a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.112230 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-5cb23fdf-98ab-40e7-9974-5df2d7e93ea2 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:36.113340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-5cb23fdf-98ab-40e7-9974-5df2d7e93ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:36.134910 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0b29dea0-8786-4065-a0de-19e2451e2a51 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:36.136310 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0b29dea0-8786-4065-a0de-19e2451e2a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:36.184735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 782/1546] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:36.186447 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484c25e-e3c3-4da0-8a47-56b3f8dcece8 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.187676 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484c25e-e3c3-4da0-8a47-56b3f8dcece8 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.187676 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484c25e-e3c3-4da0-8a47-56b3f8dcece8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.202673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 765/1547] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:36.203855 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a71b26-09b7-4183-9a5f-b6593f30f9df None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.204170 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a71b26-09b7-4183-9a5f-b6593f30f9df None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.204450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a71b26-09b7-4183-9a5f-b6593f30f9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.215347 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-4484c25e-e3c3-4da0-8a47-56b3f8dcece8 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:36.216646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-4484c25e-e3c3-4da0-8a47-56b3f8dcece8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:36.229719 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 783/1548] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:36.230704 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484c25e-e3c3-4da0-8a47-56b3f8dcece8 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:36.231766 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-34a71b26-09b7-4183-9a5f-b6593f30f9df None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:36.233040 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-34a71b26-09b7-4183-9a5f-b6593f30f9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:36.240765 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484c25e-e3c3-4da0-8a47-56b3f8dcece8 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:36.245294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 766/1549] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:36.246570 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5de53-f004-41cc-bd27-4bba0e34a6cc None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.246862 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5de53-f004-41cc-bd27-4bba0e34a6cc None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.247138 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5de53-f004-41cc-bd27-4bba0e34a6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.248026 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93743c0e-50e7-4dbf-92c3-52476b0e2eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:36.248684 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93743c0e-50e7-4dbf-92c3-52476b0e2eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.248978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93743c0e-50e7-4dbf-92c3-52476b0e2eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.249253 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93743c0e-50e7-4dbf-92c3-52476b0e2eb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.249942 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93743c0e-50e7-4dbf-92c3-52476b0e2eb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:36.251077 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: WARNING py.warnings [None req-93743c0e-50e7-4dbf-92c3-52476b0e2eb8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:32:36.251077 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: warnings.warn(msg) Aug 04 10:32:36.251077 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: Aug 04 10:32:36.252318 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93743c0e-50e7-4dbf-92c3-52476b0e2eb8 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:36.270415 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f2a5de53-f004-41cc-bd27-4bba0e34a6cc None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:36.271735 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f2a5de53-f004-41cc-bd27-4bba0e34a6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:36.276548 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 784/1550] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:32:36 2022] POST /identity/v3/users => generated 289 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:36.277735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba916f4-7387-4dd0-a91e-3937f76872f8 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.278026 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba916f4-7387-4dd0-a91e-3937f76872f8 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.278307 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba916f4-7387-4dd0-a91e-3937f76872f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.280470 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 767/1551] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:36.281304 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5de53-f004-41cc-bd27-4bba0e34a6cc None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:36.291450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5de53-f004-41cc-bd27-4bba0e34a6cc None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:36.296106 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29451d8-4812-40c0-bbb4-fc7ec54f554e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:36.296736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29451d8-4812-40c0-bbb4-fc7ec54f554e admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.297028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29451d8-4812-40c0-bbb4-fc7ec54f554e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.297307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29451d8-4812-40c0-bbb4-fc7ec54f554e admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.298050 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29451d8-4812-40c0-bbb4-fc7ec54f554e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:32:36.298363 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29451d8-4812-40c0-bbb4-fc7ec54f554e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:36.300031 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29451d8-4812-40c0-bbb4-fc7ec54f554e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:36.301254 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-4ba916f4-7387-4dd0-a91e-3937f76872f8 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:36.302613 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-4ba916f4-7387-4dd0-a91e-3937f76872f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:36.310867 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 785/1552] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:36.312126 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec0c65-7596-47d6-af22-5abb89df71dc None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.312412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec0c65-7596-47d6-af22-5abb89df71dc None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.312726 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec0c65-7596-47d6-af22-5abb89df71dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.320778 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 768/1553] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:32:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:32:36.320778 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fd6fd-8517-46a8-b541-43c42093464e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.320778 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fd6fd-8517-46a8-b541-43c42093464e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.320778 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fd6fd-8517-46a8-b541-43c42093464e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.332882 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-47ec0c65-7596-47d6-af22-5abb89df71dc None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:36.334108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-47ec0c65-7596-47d6-af22-5abb89df71dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:36.337208 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-316fd6fd-8517-46a8-b541-43c42093464e None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:36.338416 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-316fd6fd-8517-46a8-b541-43c42093464e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:36.342360 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 786/1554] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:36.343427 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac30d32-573d-4f69-8421-8dd9c57e107e None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.343719 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac30d32-573d-4f69-8421-8dd9c57e107e None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.344027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac30d32-573d-4f69-8421-8dd9c57e107e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.352011 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 769/1555] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:36.352414 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6cc60-503a-4686-9451-c169dc036843 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.353911 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6cc60-503a-4686-9451-c169dc036843 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.354207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6cc60-503a-4686-9451-c169dc036843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.364745 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-0ac30d32-573d-4f69-8421-8dd9c57e107e None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:36.365939 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-0ac30d32-573d-4f69-8421-8dd9c57e107e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:36.374061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 787/1556] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:36.375024 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac30d32-573d-4f69-8421-8dd9c57e107e None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:36.376977 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-00d6cc60-503a-4686-9451-c169dc036843 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:36.378200 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-00d6cc60-503a-4686-9451-c169dc036843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:36.385149 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac30d32-573d-4f69-8421-8dd9c57e107e None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:36.387629 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 770/1557] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:36.388589 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6cc60-503a-4686-9451-c169dc036843 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:36.390205 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09932068-18a8-4180-8959-4cad15c2f8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:36.390838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09932068-18a8-4180-8959-4cad15c2f8c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.391122 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09932068-18a8-4180-8959-4cad15c2f8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.391434 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09932068-18a8-4180-8959-4cad15c2f8c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.392185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09932068-18a8-4180-8959-4cad15c2f8c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:36.393854 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09932068-18a8-4180-8959-4cad15c2f8c4 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:36.398583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6cc60-503a-4686-9451-c169dc036843 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:36.404469 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c844f-5a96-4849-8a10-6e2da7c1adb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:36.407810 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 788/1558] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:36 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:36.408047 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c844f-5a96-4849-8a10-6e2da7c1adb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.408047 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c844f-5a96-4849-8a10-6e2da7c1adb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.408047 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c844f-5a96-4849-8a10-6e2da7c1adb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.408347 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609f4d5-e17f-4734-9d98-fd83ea4dd25b None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.408446 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c844f-5a96-4849-8a10-6e2da7c1adb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:32:36.408536 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609f4d5-e17f-4734-9d98-fd83ea4dd25b None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.408536 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609f4d5-e17f-4734-9d98-fd83ea4dd25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.408729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c844f-5a96-4849-8a10-6e2da7c1adb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:36.408729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c844f-5a96-4849-8a10-6e2da7c1adb9 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:36.421236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 771/1559] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:32:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:32:36.422256 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c844f-5a96-4849-8a10-6e2da7c1adb9 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:36.425975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-f609f4d5-e17f-4734-9d98-fd83ea4dd25b None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:36.427187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-f609f4d5-e17f-4734-9d98-fd83ea4dd25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:36.434321 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c844f-5a96-4849-8a10-6e2da7c1adb9 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:36.435289 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 789/1560] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:36.436142 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609f4d5-e17f-4734-9d98-fd83ea4dd25b None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:36.439515 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714db290-cd17-444b-b68e-30545498eab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:36.440215 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714db290-cd17-444b-b68e-30545498eab8 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.440500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714db290-cd17-444b-b68e-30545498eab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.440783 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714db290-cd17-444b-b68e-30545498eab8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.441691 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714db290-cd17-444b-b68e-30545498eab8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:36.444399 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714db290-cd17-444b-b68e-30545498eab8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:36.446248 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609f4d5-e17f-4734-9d98-fd83ea4dd25b None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:36.451890 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef468a-f2fb-4ba6-8a8e-d6149095ff5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:36.452568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef468a-f2fb-4ba6-8a8e-d6149095ff5e admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.452857 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef468a-f2fb-4ba6-8a8e-d6149095ff5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.453170 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef468a-f2fb-4ba6-8a8e-d6149095ff5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.453904 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef468a-f2fb-4ba6-8a8e-d6149095ff5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:32:36.454186 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef468a-f2fb-4ba6-8a8e-d6149095ff5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:36.455259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: WARNING py.warnings [None req-dcef468a-f2fb-4ba6-8a8e-d6149095ff5e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:32:36.455259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: warnings.warn(msg) Aug 04 10:32:36.455259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: Aug 04 10:32:36.456181 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef468a-f2fb-4ba6-8a8e-d6149095ff5e admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:36.469119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 790/1561] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:32:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:32:36.471045 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25276e0-a370-4b2f-a7af-d0a1d775d2cc None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.471340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25276e0-a370-4b2f-a7af-d0a1d775d2cc None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.471755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25276e0-a370-4b2f-a7af-d0a1d775d2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.475500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 772/1562] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:32:36 2022] POST /identity/v3/projects => generated 376 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:36.476420 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714db290-cd17-444b-b68e-30545498eab8 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:36.498469 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-c25276e0-a370-4b2f-a7af-d0a1d775d2cc None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:36.499755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c25276e0-a370-4b2f-a7af-d0a1d775d2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:36.526909 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:36.553934 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714db290-cd17-444b-b68e-30545498eab8 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:36.559077 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae76df-8ee8-4489-8a3a-9059be54535e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:36.559733 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae76df-8ee8-4489-8a3a-9059be54535e admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.560196 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae76df-8ee8-4489-8a3a-9059be54535e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.560508 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae76df-8ee8-4489-8a3a-9059be54535e admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.561235 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae76df-8ee8-4489-8a3a-9059be54535e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:32:36.561549 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae76df-8ee8-4489-8a3a-9059be54535e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:36.563205 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae76df-8ee8-4489-8a3a-9059be54535e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:36.568571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 791/1563] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:36.569488 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25276e0-a370-4b2f-a7af-d0a1d775d2cc None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:36.584098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 773/1564] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:32:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:32:36.584346 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25276e0-a370-4b2f-a7af-d0a1d775d2cc None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:36.584468 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae76df-8ee8-4489-8a3a-9059be54535e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:36.587579 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22c32f-679b-4284-8813-bbb5b4fea5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:36.587890 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22c32f-679b-4284-8813-bbb5b4fea5f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.588230 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22c32f-679b-4284-8813-bbb5b4fea5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.588517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22c32f-679b-4284-8813-bbb5b4fea5f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.589241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22c32f-679b-4284-8813-bbb5b4fea5f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:32:36.589545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22c32f-679b-4284-8813-bbb5b4fea5f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:36.591212 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22c32f-679b-4284-8813-bbb5b4fea5f8 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:36.594095 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:36.597534 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae76df-8ee8-4489-8a3a-9059be54535e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:36.604062 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea3251-a477-4517-baf4-b16ba20683af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:36.604713 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea3251-a477-4517-baf4-b16ba20683af admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.605010 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea3251-a477-4517-baf4-b16ba20683af admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.605288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea3251-a477-4517-baf4-b16ba20683af admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.606038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea3251-a477-4517-baf4-b16ba20683af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:36.607664 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea3251-a477-4517-baf4-b16ba20683af admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:36.608010 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 792/1565] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:32:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:32:36.608843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22c32f-679b-4284-8813-bbb5b4fea5f8 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:36.620085 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22c32f-679b-4284-8813-bbb5b4fea5f8 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:36.624874 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9d62e-d88a-48da-81ad-9e557515fb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:36.625520 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9d62e-d88a-48da-81ad-9e557515fb2a admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.626004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9d62e-d88a-48da-81ad-9e557515fb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.626401 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9d62e-d88a-48da-81ad-9e557515fb2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.626969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda9d62e-d88a-48da-81ad-9e557515fb2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:36.629045 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda9d62e-d88a-48da-81ad-9e557515fb2a admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:36.630661 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 774/1566] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:36 2022] GET /identity/v3/roles => generated 1427 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:36.631553 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea3251-a477-4517-baf4-b16ba20683af admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:36.641927 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea3251-a477-4517-baf4-b16ba20683af admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:36.647009 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f6283-90f8-4680-a343-1514e29d256c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:36.647651 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f6283-90f8-4680-a343-1514e29d256c admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.647932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f6283-90f8-4680-a343-1514e29d256c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.648256 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f6283-90f8-4680-a343-1514e29d256c admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.648955 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f6283-90f8-4680-a343-1514e29d256c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:32:36.649260 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f6283-90f8-4680-a343-1514e29d256c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:36.650925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f6283-90f8-4680-a343-1514e29d256c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:36.664182 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 775/1567] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:32:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:32:36.664806 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f6283-90f8-4680-a343-1514e29d256c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:36.677236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f6283-90f8-4680-a343-1514e29d256c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:36.682403 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 793/1568] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:32:36 2022] POST /identity/v3/projects => generated 382 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:36.682403 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9d62e-d88a-48da-81ad-9e557515fb2a admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:36.740694 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989c26e-c7bb-4c30-8b06-240ec3d69fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:36.741436 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989c26e-c7bb-4c30-8b06-240ec3d69fd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.741838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989c26e-c7bb-4c30-8b06-240ec3d69fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.742515 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989c26e-c7bb-4c30-8b06-240ec3d69fd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.742966 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989c26e-c7bb-4c30-8b06-240ec3d69fd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:36.744962 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989c26e-c7bb-4c30-8b06-240ec3d69fd2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:36.748126 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9d62e-d88a-48da-81ad-9e557515fb2a admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:36.752966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80928574-9050-4154-a94c-094edd3aee10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:36.753644 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80928574-9050-4154-a94c-094edd3aee10 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.753928 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80928574-9050-4154-a94c-094edd3aee10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.754203 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80928574-9050-4154-a94c-094edd3aee10 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.754883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80928574-9050-4154-a94c-094edd3aee10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:36.756898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80928574-9050-4154-a94c-094edd3aee10 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:36.778388 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 776/1569] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:32:36 2022] POST /identity/v3/projects => generated 384 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:36.780733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 794/1570] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:32:36 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:36.783391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989c26e-c7bb-4c30-8b06-240ec3d69fd2 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:36.783536 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80928574-9050-4154-a94c-094edd3aee10 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:36.799403 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80928574-9050-4154-a94c-094edd3aee10 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:36.851574 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989c26e-c7bb-4c30-8b06-240ec3d69fd2 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:36.857029 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c0bf8-a2c9-4214-a79b-6aa9b84ebcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:36.857683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c0bf8-a2c9-4214-a79b-6aa9b84ebcd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.857990 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c0bf8-a2c9-4214-a79b-6aa9b84ebcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.858253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c0bf8-a2c9-4214-a79b-6aa9b84ebcd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.859037 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c0bf8-a2c9-4214-a79b-6aa9b84ebcd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:32:36.859279 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c0bf8-a2c9-4214-a79b-6aa9b84ebcd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:36.861137 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c0bf8-a2c9-4214-a79b-6aa9b84ebcd2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:36.864235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd21ad0-60eb-4530-9f02-e0fcf27a3f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:36.864881 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd21ad0-60eb-4530-9f02-e0fcf27a3f41 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.865162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd21ad0-60eb-4530-9f02-e0fcf27a3f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.865436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd21ad0-60eb-4530-9f02-e0fcf27a3f41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.866135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd21ad0-60eb-4530-9f02-e0fcf27a3f41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:36.868139 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd21ad0-60eb-4530-9f02-e0fcf27a3f41 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:36.874207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 777/1571] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:32:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:32:36.874986 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c0bf8-a2c9-4214-a79b-6aa9b84ebcd2 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:36.885747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c0bf8-a2c9-4214-a79b-6aa9b84ebcd2 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:36.890833 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc39151-fc67-4a0d-b377-8c80340638a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:36.891668 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc39151-fc67-4a0d-b377-8c80340638a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:36.892007 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc39151-fc67-4a0d-b377-8c80340638a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:36.892328 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc39151-fc67-4a0d-b377-8c80340638a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:36.893087 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc39151-fc67-4a0d-b377-8c80340638a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:36.895279 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc39151-fc67-4a0d-b377-8c80340638a1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:36.903264 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 795/1572] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:32:36 2022] POST /identity/v3/projects => generated 374 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:36.907760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd21ad0-60eb-4530-9f02-e0fcf27a3f41 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:36.931711 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 778/1573] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:32:36 2022] POST /identity/v3/projects => generated 390 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:36.933183 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc39151-fc67-4a0d-b377-8c80340638a1 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:36.939033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:36.970032 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:36.989444 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd21ad0-60eb-4530-9f02-e0fcf27a3f41 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.005975 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc39151-fc67-4a0d-b377-8c80340638a1 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.010854 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1302bbe-7f5a-4d46-9420-c36eb06dd5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.011551 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1302bbe-7f5a-4d46-9420-c36eb06dd5c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.011837 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1302bbe-7f5a-4d46-9420-c36eb06dd5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.012149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1302bbe-7f5a-4d46-9420-c36eb06dd5c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.012836 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1302bbe-7f5a-4d46-9420-c36eb06dd5c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.014807 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1302bbe-7f5a-4d46-9420-c36eb06dd5c9 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.047299 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 779/1574] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:32:36 2022] POST /identity/v3/projects => generated 380 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:37.048731 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1302bbe-7f5a-4d46-9420-c36eb06dd5c9 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:37.060312 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6e997-1377-4095-bb02-6f22a2961f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.062526 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6e997-1377-4095-bb02-6f22a2961f41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.062526 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6e997-1377-4095-bb02-6f22a2961f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.062526 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6e997-1377-4095-bb02-6f22a2961f41 admin admin] PATH_INFO: `/v3/projects/6f46ab11dd0b428486c673afc14b5b63/users/3aa2e2a5a6ed4c5c9db91269266dd30b/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.086571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6e997-1377-4095-bb02-6f22a2961f41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f46ab11dd0b428486c673afc14b5b63, user_id=3aa2e2a5a6ed4c5c9db91269266dd30b, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.089350 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6e997-1377-4095-bb02-6f22a2961f41 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.106765 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 796/1575] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:32:36 2022] PUT /identity/v3/projects/6f46ab11dd0b428486c673afc14b5b63/users/3aa2e2a5a6ed4c5c9db91269266dd30b/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:37.107295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6e997-1377-4095-bb02-6f22a2961f41 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:37.128476 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1302bbe-7f5a-4d46-9420-c36eb06dd5c9 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.183128 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6e997-1377-4095-bb02-6f22a2961f41 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.188917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def0eb5-f15a-4a3c-a9f6-4e2b0a326a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.189595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def0eb5-f15a-4a3c-a9f6-4e2b0a326a57 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.189873 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def0eb5-f15a-4a3c-a9f6-4e2b0a326a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.190141 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def0eb5-f15a-4a3c-a9f6-4e2b0a326a57 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.190821 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def0eb5-f15a-4a3c-a9f6-4e2b0a326a57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.192802 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def0eb5-f15a-4a3c-a9f6-4e2b0a326a57 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.192990 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c311c9a7-d48b-4e3d-9abc-6c4955646cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.193638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311c9a7-d48b-4e3d-9abc-6c4955646cdd admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.193928 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311c9a7-d48b-4e3d-9abc-6c4955646cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.194217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311c9a7-d48b-4e3d-9abc-6c4955646cdd admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.194898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c311c9a7-d48b-4e3d-9abc-6c4955646cdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.196899 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c311c9a7-d48b-4e3d-9abc-6c4955646cdd admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.220827 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 797/1576] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:32:37 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:37.222049 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def0eb5-f15a-4a3c-a9f6-4e2b0a326a57 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:37.223512 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 780/1577] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:32:37 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:37.224721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c311c9a7-d48b-4e3d-9abc-6c4955646cdd admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:37.234519 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def0eb5-f15a-4a3c-a9f6-4e2b0a326a57 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.235472 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c311c9a7-d48b-4e3d-9abc-6c4955646cdd admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.239600 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9497e7e-167f-464b-979b-3f65fd63259d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.240279 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9497e7e-167f-464b-979b-3f65fd63259d admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.240626 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9497e7e-167f-464b-979b-3f65fd63259d admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.240933 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9497e7e-167f-464b-979b-3f65fd63259d admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.241106 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f449e-7b20-43de-86b7-e846ada9e229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.241531 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f449e-7b20-43de-86b7-e846ada9e229 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.241715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9497e7e-167f-464b-979b-3f65fd63259d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.241868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f449e-7b20-43de-86b7-e846ada9e229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.242186 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f449e-7b20-43de-86b7-e846ada9e229 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.242747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851f449e-7b20-43de-86b7-e846ada9e229 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.243204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9497e7e-167f-464b-979b-3f65fd63259d admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.244714 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851f449e-7b20-43de-86b7-e846ada9e229 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.255021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 798/1578] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:37 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:37.255890 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9497e7e-167f-464b-979b-3f65fd63259d admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:37.266294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9497e7e-167f-464b-979b-3f65fd63259d admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.271627 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4589d3-ad13-4cb8-898f-8cbeb5c106e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.272395 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4589d3-ad13-4cb8-898f-8cbeb5c106e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.272678 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4589d3-ad13-4cb8-898f-8cbeb5c106e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.272960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4589d3-ad13-4cb8-898f-8cbeb5c106e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.273659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4589d3-ad13-4cb8-898f-8cbeb5c106e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.275596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4589d3-ad13-4cb8-898f-8cbeb5c106e6 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.279488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 781/1579] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:32:37 2022] POST /identity/v3/projects => generated 380 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:37.280427 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f449e-7b20-43de-86b7-e846ada9e229 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:37.286648 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:37.299183 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 799/1580] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:32:37 2022] POST /identity/v3/users => generated 288 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:37.303253 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4589d3-ad13-4cb8-898f-8cbeb5c106e6 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:37.313368 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4589d3-ad13-4cb8-898f-8cbeb5c106e6 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.353614 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:37.366039 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f449e-7b20-43de-86b7-e846ada9e229 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.370958 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7dabd7-ea41-45f7-aca2-9122204e1a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.371593 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7dabd7-ea41-45f7-aca2-9122204e1a1c admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.371873 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7dabd7-ea41-45f7-aca2-9122204e1a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.372174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7dabd7-ea41-45f7-aca2-9122204e1a1c admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.372844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7dabd7-ea41-45f7-aca2-9122204e1a1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.374781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7dabd7-ea41-45f7-aca2-9122204e1a1c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.379529 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105d647-ad83-48ca-b73a-8cfccc23c568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.380150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105d647-ad83-48ca-b73a-8cfccc23c568 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.380600 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105d647-ad83-48ca-b73a-8cfccc23c568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.380881 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105d647-ad83-48ca-b73a-8cfccc23c568 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.381558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7105d647-ad83-48ca-b73a-8cfccc23c568 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.383760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7105d647-ad83-48ca-b73a-8cfccc23c568 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.400287 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 782/1581] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:32:37 2022] POST /identity/v3/users => generated 296 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:37.401866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc17c73-58d3-4a89-8236-25a5f5a6d34c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.402153 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc17c73-58d3-4a89-8236-25a5f5a6d34c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.402429 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc17c73-58d3-4a89-8236-25a5f5a6d34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.409636 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 800/1582] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:32:37 2022] POST /identity/v3/users => generated 291 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:37.411528 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105d647-ad83-48ca-b73a-8cfccc23c568 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:37.421906 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-5fc17c73-58d3-4a89-8236-25a5f5a6d34c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:37.422119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105d647-ad83-48ca-b73a-8cfccc23c568 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.423258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5fc17c73-58d3-4a89-8236-25a5f5a6d34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:37.426886 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e926140-5d51-4b1b-a333-ce76c58d221b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.427514 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e926140-5d51-4b1b-a333-ce76c58d221b admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.427804 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e926140-5d51-4b1b-a333-ce76c58d221b admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.428121 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e926140-5d51-4b1b-a333-ce76c58d221b admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.428835 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e926140-5d51-4b1b-a333-ce76c58d221b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.430468 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e926140-5d51-4b1b-a333-ce76c58d221b admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.431174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 783/1583] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:37 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:37.434570 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc17c73-58d3-4a89-8236-25a5f5a6d34c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:37.442979 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 801/1584] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:37 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:37.443905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc17c73-58d3-4a89-8236-25a5f5a6d34c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.444080 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e926140-5d51-4b1b-a333-ce76c58d221b admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:37.451601 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabab674-5b54-440f-92c4-1c7c40835f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.452524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabab674-5b54-440f-92c4-1c7c40835f66 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.452806 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabab674-5b54-440f-92c4-1c7c40835f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.453076 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabab674-5b54-440f-92c4-1c7c40835f66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.453771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabab674-5b54-440f-92c4-1c7c40835f66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.455485 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabab674-5b54-440f-92c4-1c7c40835f66 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.456301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e926140-5d51-4b1b-a333-ce76c58d221b admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.461013 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e49415-a193-4a9f-bc3f-ad58a8a0ba95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.461662 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e49415-a193-4a9f-bc3f-ad58a8a0ba95 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.461961 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e49415-a193-4a9f-bc3f-ad58a8a0ba95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.462236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e49415-a193-4a9f-bc3f-ad58a8a0ba95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.462933 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e49415-a193-4a9f-bc3f-ad58a8a0ba95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.464705 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e49415-a193-4a9f-bc3f-ad58a8a0ba95 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.466563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 784/1585] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:37 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:37.467234 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabab674-5b54-440f-92c4-1c7c40835f66 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:37.477792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabab674-5b54-440f-92c4-1c7c40835f66 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.480362 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 802/1586] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:37 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:37.484109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e49415-a193-4a9f-bc3f-ad58a8a0ba95 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:37.486810 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993ee2c-645f-4a5e-a81d-bea3d17bcd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.488114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993ee2c-645f-4a5e-a81d-bea3d17bcd99 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.488380 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993ee2c-645f-4a5e-a81d-bea3d17bcd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.488665 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993ee2c-645f-4a5e-a81d-bea3d17bcd99 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.489351 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993ee2c-645f-4a5e-a81d-bea3d17bcd99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.491308 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993ee2c-645f-4a5e-a81d-bea3d17bcd99 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.495019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e49415-a193-4a9f-bc3f-ad58a8a0ba95 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.499774 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ff57b-efcb-4a3c-ae13-5159f1be99ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.500884 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ff57b-efcb-4a3c-ae13-5159f1be99ea admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.500884 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ff57b-efcb-4a3c-ae13-5159f1be99ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.501103 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ff57b-efcb-4a3c-ae13-5159f1be99ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.501937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026ff57b-efcb-4a3c-ae13-5159f1be99ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.503537 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026ff57b-efcb-4a3c-ae13-5159f1be99ea admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.515789 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 785/1587] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:32:37 2022] POST /identity/v3/users => generated 291 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:37.519169 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 803/1588] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:37 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:37.519169 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ff57b-efcb-4a3c-ae13-5159f1be99ea admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:37.519405 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993ee2c-645f-4a5e-a81d-bea3d17bcd99 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:37.528175 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993ee2c-645f-4a5e-a81d-bea3d17bcd99 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.528915 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ff57b-efcb-4a3c-ae13-5159f1be99ea admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.534125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f801ed5-09aa-4e22-bf84-a6383fcdccf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.534841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014983c-7532-47c1-a227-4e96a68ea52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.534993 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f801ed5-09aa-4e22-bf84-a6383fcdccf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.535094 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f801ed5-09aa-4e22-bf84-a6383fcdccf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.535339 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f801ed5-09aa-4e22-bf84-a6383fcdccf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.535492 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014983c-7532-47c1-a227-4e96a68ea52c admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.535774 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014983c-7532-47c1-a227-4e96a68ea52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.536069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014983c-7532-47c1-a227-4e96a68ea52c admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.536190 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f801ed5-09aa-4e22-bf84-a6383fcdccf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.536892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6014983c-7532-47c1-a227-4e96a68ea52c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.537752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f801ed5-09aa-4e22-bf84-a6383fcdccf7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.538667 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6014983c-7532-47c1-a227-4e96a68ea52c admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.550017 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 804/1589] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:37 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:37.550488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 786/1590] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:37 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:37.551166 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014983c-7532-47c1-a227-4e96a68ea52c admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:37.555990 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f801ed5-09aa-4e22-bf84-a6383fcdccf7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:37.564430 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f801ed5-09aa-4e22-bf84-a6383fcdccf7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.569419 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799e0f9-942d-4179-8999-8a436f76447a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.570025 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799e0f9-942d-4179-8999-8a436f76447a admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.570335 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799e0f9-942d-4179-8999-8a436f76447a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.570592 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799e0f9-942d-4179-8999-8a436f76447a admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.571300 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799e0f9-942d-4179-8999-8a436f76447a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.573786 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799e0f9-942d-4179-8999-8a436f76447a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.584203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 787/1591] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:37 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:37.585296 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799e0f9-942d-4179-8999-8a436f76447a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:37.595136 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799e0f9-942d-4179-8999-8a436f76447a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.602890 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418d039-9ff9-4e14-9f98-6d85e3e57736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.603826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418d039-9ff9-4e14-9f98-6d85e3e57736 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.604153 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418d039-9ff9-4e14-9f98-6d85e3e57736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.604426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418d039-9ff9-4e14-9f98-6d85e3e57736 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.605119 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418d039-9ff9-4e14-9f98-6d85e3e57736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.606737 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418d039-9ff9-4e14-9f98-6d85e3e57736 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.617370 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 788/1592] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:37 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:37.617685 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014983c-7532-47c1-a227-4e96a68ea52c admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.618174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418d039-9ff9-4e14-9f98-6d85e3e57736 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:37.621909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4cd81-46d4-4758-9fd7-28d12148ff53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.622649 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4cd81-46d4-4758-9fd7-28d12148ff53 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.623011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4cd81-46d4-4758-9fd7-28d12148ff53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.623367 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4cd81-46d4-4758-9fd7-28d12148ff53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.631862 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418d039-9ff9-4e14-9f98-6d85e3e57736 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.633532 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de4cd81-46d4-4758-9fd7-28d12148ff53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.635013 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de4cd81-46d4-4758-9fd7-28d12148ff53 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.637331 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c549dbf4-e2f3-45b6-81e3-0588c32e1d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.638043 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549dbf4-e2f3-45b6-81e3-0588c32e1d85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.638617 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549dbf4-e2f3-45b6-81e3-0588c32e1d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.638975 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549dbf4-e2f3-45b6-81e3-0588c32e1d85 admin admin] PATH_INFO: `/v3/projects/5d18be11fe1d499dac0af5ee521d0203/users/7631e70ced3b4e72b64d971d644fcab2/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.647110 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 805/1593] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:32:37 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:37.648051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4cd81-46d4-4758-9fd7-28d12148ff53 service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:37.659871 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4cd81-46d4-4758-9fd7-28d12148ff53 service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.666653 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b01344-89db-4fcf-8558-6cf12720d69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.666946 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c549dbf4-e2f3-45b6-81e3-0588c32e1d85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d18be11fe1d499dac0af5ee521d0203, user_id=7631e70ced3b4e72b64d971d644fcab2, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.667906 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: WARNING py.warnings [None req-c549dbf4-e2f3-45b6-81e3-0588c32e1d85 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:32:37.667906 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: warnings.warn(msg) Aug 04 10:32:37.667906 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: Aug 04 10:32:37.668508 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b01344-89db-4fcf-8558-6cf12720d69e admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.668877 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b01344-89db-4fcf-8558-6cf12720d69e admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.669321 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b01344-89db-4fcf-8558-6cf12720d69e admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.670169 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c549dbf4-e2f3-45b6-81e3-0588c32e1d85 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.670405 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b01344-89db-4fcf-8558-6cf12720d69e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.673152 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b01344-89db-4fcf-8558-6cf12720d69e admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.693041 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 806/1594] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:37 2022] GET /identity/v3/roles => generated 1427 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:37.694517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b01344-89db-4fcf-8558-6cf12720d69e admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:37.711991 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 789/1595] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:32:37 2022] PUT /identity/v3/projects/5d18be11fe1d499dac0af5ee521d0203/users/7631e70ced3b4e72b64d971d644fcab2/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:37.711991 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c549dbf4-e2f3-45b6-81e3-0588c32e1d85 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:37.712503 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b01344-89db-4fcf-8558-6cf12720d69e admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.794433 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593dfd3a-eb57-48fd-8c2e-ab306a1a11cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.795130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dfd3a-eb57-48fd-8c2e-ab306a1a11cc admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.795438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dfd3a-eb57-48fd-8c2e-ab306a1a11cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.795750 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dfd3a-eb57-48fd-8c2e-ab306a1a11cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.796560 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593dfd3a-eb57-48fd-8c2e-ab306a1a11cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.798388 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593dfd3a-eb57-48fd-8c2e-ab306a1a11cc admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.799177 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c549dbf4-e2f3-45b6-81e3-0588c32e1d85 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.805880 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0f08b-4acc-4851-90de-0cea68d06209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.806525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0f08b-4acc-4851-90de-0cea68d06209 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.806998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0f08b-4acc-4851-90de-0cea68d06209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.807278 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0f08b-4acc-4851-90de-0cea68d06209 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.808016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0f08b-4acc-4851-90de-0cea68d06209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.809727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0f08b-4acc-4851-90de-0cea68d06209 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.811380 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 807/1596] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:37 2022] GET /identity/v3/roles => generated 1427 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:37.815013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:37.816300 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593dfd3a-eb57-48fd-8c2e-ab306a1a11cc admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:37.824740 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 790/1597] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:37 2022] GET /identity/v3/roles => generated 1427 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:37.829217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593dfd3a-eb57-48fd-8c2e-ab306a1a11cc admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.832368 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0f08b-4acc-4851-90de-0cea68d06209 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:37.838212 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31408f2-93ce-406f-b417-fbfc20683126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.839247 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31408f2-93ce-406f-b417-fbfc20683126 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.839783 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31408f2-93ce-406f-b417-fbfc20683126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.840319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31408f2-93ce-406f-b417-fbfc20683126 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.841309 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31408f2-93ce-406f-b417-fbfc20683126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.843286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31408f2-93ce-406f-b417-fbfc20683126 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.849791 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0f08b-4acc-4851-90de-0cea68d06209 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.850336 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:37.857039 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d340e-1f28-4987-8e22-2905487777b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.857288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d340e-1f28-4987-8e22-2905487777b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.857394 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 808/1598] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:37 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:37.858195 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31408f2-93ce-406f-b417-fbfc20683126 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:37.860381 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d340e-1f28-4987-8e22-2905487777b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.861023 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d340e-1f28-4987-8e22-2905487777b0 admin admin] PATH_INFO: `/v3/projects/c95096b8b71b4441836e289040285c19/users/2ac28f9efd59480c8c6f5385e5945b25/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.871890 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31408f2-93ce-406f-b417-fbfc20683126 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:37.878010 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2176e600-83f3-48d5-9a7e-f7739640d345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:37.879122 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176e600-83f3-48d5-9a7e-f7739640d345 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.879122 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176e600-83f3-48d5-9a7e-f7739640d345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.879490 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176e600-83f3-48d5-9a7e-f7739640d345 admin admin] PATH_INFO: `/v3/projects/051b6a3defca48d7b4f8bb6705f89f20/users/41a38b5af1654f44bc6cb5832c2e8e87/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.881085 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d340e-1f28-4987-8e22-2905487777b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c95096b8b71b4441836e289040285c19, user_id=2ac28f9efd59480c8c6f5385e5945b25, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.884116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d340e-1f28-4987-8e22-2905487777b0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.900869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2176e600-83f3-48d5-9a7e-f7739640d345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=051b6a3defca48d7b4f8bb6705f89f20, user_id=41a38b5af1654f44bc6cb5832c2e8e87, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:37.902618 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 791/1599] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:32:37 2022] PUT /identity/v3/projects/c95096b8b71b4441836e289040285c19/users/2ac28f9efd59480c8c6f5385e5945b25/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:37.904103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d340e-1f28-4987-8e22-2905487777b0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:37.907417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2176e600-83f3-48d5-9a7e-f7739640d345 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:37.939224 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 809/1600] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:32:37 2022] PUT /identity/v3/projects/051b6a3defca48d7b4f8bb6705f89f20/users/41a38b5af1654f44bc6cb5832c2e8e87/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:37.939224 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225f7fc-6954-4674-9fbf-84d9625dfc51 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:37.939713 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225f7fc-6954-4674-9fbf-84d9625dfc51 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:37.940222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225f7fc-6954-4674-9fbf-84d9625dfc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:37.990113 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-1225f7fc-6954-4674-9fbf-84d9625dfc51 None None] MFA Rules not processed for user `7631e70ced3b4e72b64d971d644fcab2`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:37.991347 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-1225f7fc-6954-4674-9fbf-84d9625dfc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:37.994714 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d340e-1f28-4987-8e22-2905487777b0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:38.070877 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 810/1601] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:38.072372 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225f7fc-6954-4674-9fbf-84d9625dfc51 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:38.088345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c25512-cb19-4b4e-afbf-da310860347e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:38.089196 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c25512-cb19-4b4e-afbf-da310860347e admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:38.089500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c25512-cb19-4b4e-afbf-da310860347e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:38.089837 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c25512-cb19-4b4e-afbf-da310860347e admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:38.090784 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c25512-cb19-4b4e-afbf-da310860347e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:38.092662 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c25512-cb19-4b4e-afbf-da310860347e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:38.105393 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 792/1602] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:37 2022] GET /identity/v3/roles => generated 1427 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:38.106365 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c25512-cb19-4b4e-afbf-da310860347e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:38.120024 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c25512-cb19-4b4e-afbf-da310860347e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:38.125723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4fe54-7f38-4dc9-8b23-64fa3695383d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:38.127217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4fe54-7f38-4dc9-8b23-64fa3695383d admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:38.127217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4fe54-7f38-4dc9-8b23-64fa3695383d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:38.127589 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4fe54-7f38-4dc9-8b23-64fa3695383d admin admin] PATH_INFO: `/v3/projects/e2c1805521bc43f8a663bc01eeddda2f/users/e8fc011e92854d219da5f0f8a5fba798/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:38.153170 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc4fe54-7f38-4dc9-8b23-64fa3695383d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2c1805521bc43f8a663bc01eeddda2f, user_id=e8fc011e92854d219da5f0f8a5fba798, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:38.156449 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc4fe54-7f38-4dc9-8b23-64fa3695383d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:38.175450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 793/1603] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:32:38 2022] PUT /identity/v3/projects/e2c1805521bc43f8a663bc01eeddda2f/users/e8fc011e92854d219da5f0f8a5fba798/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:38.177075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4fe54-7f38-4dc9-8b23-64fa3695383d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:38.192532 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225f7fc-6954-4674-9fbf-84d9625dfc51 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:38.255667 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:38.274323 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4fe54-7f38-4dc9-8b23-64fa3695383d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:38.280236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2e1d7-ee35-43bc-b89c-ad89e04b6677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:38.281138 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2e1d7-ee35-43bc-b89c-ad89e04b6677 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:38.281419 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2e1d7-ee35-43bc-b89c-ad89e04b6677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:38.281697 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2e1d7-ee35-43bc-b89c-ad89e04b6677 admin admin] PATH_INFO: `/v3/projects/7121bd118a2549c39332cfdb7edaa92f/users/3170dc3323f54512bc8819c8efa86b0f/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:38.282526 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb491c4b-14f3-4e73-87a6-d05d5f4cfa7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:38.283571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb491c4b-14f3-4e73-87a6-d05d5f4cfa7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:38.283697 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb491c4b-14f3-4e73-87a6-d05d5f4cfa7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:38.283909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb491c4b-14f3-4e73-87a6-d05d5f4cfa7c admin admin] PATH_INFO: `/v3/projects/c1dbefa7237d4118a80cac6078fd2023/users/4ec1e2d4753d47d2a565485c65948746/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:38.300271 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace2e1d7-ee35-43bc-b89c-ad89e04b6677 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7121bd118a2549c39332cfdb7edaa92f, user_id=3170dc3323f54512bc8819c8efa86b0f, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:38.303144 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb491c4b-14f3-4e73-87a6-d05d5f4cfa7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1dbefa7237d4118a80cac6078fd2023, user_id=4ec1e2d4753d47d2a565485c65948746, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:38.303456 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace2e1d7-ee35-43bc-b89c-ad89e04b6677 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:38.306314 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb491c4b-14f3-4e73-87a6-d05d5f4cfa7c admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:38.309890 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:38.322232 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 794/1604] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:32:38 2022] PUT /identity/v3/projects/7121bd118a2549c39332cfdb7edaa92f/users/3170dc3323f54512bc8819c8efa86b0f/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:38.323888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351b693-39ba-414b-95e8-80e68c42f776 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:38.324205 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351b693-39ba-414b-95e8-80e68c42f776 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:38.324482 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351b693-39ba-414b-95e8-80e68c42f776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:38.328708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 811/1605] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:32:38 2022] PUT /identity/v3/projects/c1dbefa7237d4118a80cac6078fd2023/users/4ec1e2d4753d47d2a565485c65948746/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:38.334750 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c49115-752a-4334-91b9-48e7af004dc0 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:38.334750 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c49115-752a-4334-91b9-48e7af004dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:38.334750 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c49115-752a-4334-91b9-48e7af004dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:38.359908 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-a8c49115-752a-4334-91b9-48e7af004dc0 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:38.361374 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-a8c49115-752a-4334-91b9-48e7af004dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:38.370056 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-3351b693-39ba-414b-95e8-80e68c42f776 None None] MFA Rules not processed for user `2ac28f9efd59480c8c6f5385e5945b25`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:38.371563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3351b693-39ba-414b-95e8-80e68c42f776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:38.469207 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 812/1606] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:38 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:38.470592 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c49115-752a-4334-91b9-48e7af004dc0 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:38.479335 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 795/1607] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:38.479335 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4ef90-9da5-4d34-85ed-0db50b03a8f9 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:38.479335 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4ef90-9da5-4d34-85ed-0db50b03a8f9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:38.479335 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4ef90-9da5-4d34-85ed-0db50b03a8f9 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:38.486263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 796/1608] 10.222.0.121 () {54 vars in 967 bytes} [Thu Aug 4 10:32:38 2022] GET /identity => generated 271 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:32:38.487842 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65bbc0-a6c5-42a2-a410-b322f4125473 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:38.488171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65bbc0-a6c5-42a2-a410-b322f4125473 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:38.488585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65bbc0-a6c5-42a2-a410-b322f4125473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:38.490919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c49115-752a-4334-91b9-48e7af004dc0 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:38.508275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5b558-e7db-4d63-bbc6-7ab810825360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:38.510451 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5b558-e7db-4d63-bbc6-7ab810825360 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:38.511696 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5b558-e7db-4d63-bbc6-7ab810825360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:38.512261 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5b558-e7db-4d63-bbc6-7ab810825360 admin admin] PATH_INFO: `/v3/projects/c167faad577d47dfaf0d010a3f983fe3/users/9520a49687b24281b78f08d43cb27dc3/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:38.562626 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-bf65bbc0-a6c5-42a2-a410-b322f4125473 None None] MFA Rules not processed for user `e8fc011e92854d219da5f0f8a5fba798`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:38.564285 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bf65bbc0-a6c5-42a2-a410-b322f4125473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:38.566998 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5b558-e7db-4d63-bbc6-7ab810825360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c167faad577d47dfaf0d010a3f983fe3, user_id=9520a49687b24281b78f08d43cb27dc3, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:38.572426 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5b558-e7db-4d63-bbc6-7ab810825360 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:38.605453 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 813/1609] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:32:38 2022] PUT /identity/v3/projects/c167faad577d47dfaf0d010a3f983fe3/users/9520a49687b24281b78f08d43cb27dc3/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:38.610909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461d161-f306-46fa-8c7a-2ba5e04a60cd None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:38.611204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461d161-f306-46fa-8c7a-2ba5e04a60cd None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:38.611473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461d161-f306-46fa-8c7a-2ba5e04a60cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:38.633119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-1461d161-f306-46fa-8c7a-2ba5e04a60cd None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:38.634658 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-1461d161-f306-46fa-8c7a-2ba5e04a60cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:38.703068 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 797/1610] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:38.704738 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d7d3a-9f66-4df7-be46-07c05e34a9b7 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:38.705315 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d7d3a-9f66-4df7-be46-07c05e34a9b7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:38.705905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d7d3a-9f66-4df7-be46-07c05e34a9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:38.726953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 814/1611] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:38 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:38.729037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3ee58-428b-4e4e-8f29-538ae455b7a6 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:38.729370 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3ee58-428b-4e4e-8f29-538ae455b7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:38.730088 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3ee58-428b-4e4e-8f29-538ae455b7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:38.741142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-652d7d3a-9f66-4df7-be46-07c05e34a9b7 None None] MFA Rules not processed for user `4ec1e2d4753d47d2a565485c65948746`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:38.742551 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-652d7d3a-9f66-4df7-be46-07c05e34a9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:38.781940 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-5ab3ee58-428b-4e4e-8f29-538ae455b7a6 None None] MFA Rules not processed for user `0d947aeb749546309a8d4b7938087ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:38.785030 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-5ab3ee58-428b-4e4e-8f29-538ae455b7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:38.795753 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:38.833011 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 798/1612] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:38.834008 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d7d3a-9f66-4df7-be46-07c05e34a9b7 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:38.878251 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:38.902392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 815/1613] 10.222.0.121 () {58 vars in 1064 bytes} [Thu Aug 4 10:32:38 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:38.904656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698bdda-8b28-435b-ada8-d736dfc6b7e8 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:38.905183 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698bdda-8b28-435b-ada8-d736dfc6b7e8 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:38.905697 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698bdda-8b28-435b-ada8-d736dfc6b7e8 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:38.915144 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 816/1614] 10.222.0.121 () {54 vars in 967 bytes} [Thu Aug 4 10:32:38 2022] GET /identity => generated 271 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:32:38.917396 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c3625-4bf2-49c3-b49d-0a68c805c004 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:38.917562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c3625-4bf2-49c3-b49d-0a68c805c004 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:38.917887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c3625-4bf2-49c3-b49d-0a68c805c004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:38.918372 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d7d3a-9f66-4df7-be46-07c05e34a9b7 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:38.922764 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfda8f9-6960-4b2c-af75-77c14ab27ef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:38.923744 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfda8f9-6960-4b2c-af75-77c14ab27ef5 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:38.924292 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfda8f9-6960-4b2c-af75-77c14ab27ef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:38.924756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfda8f9-6960-4b2c-af75-77c14ab27ef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:38.936675 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfda8f9-6960-4b2c-af75-77c14ab27ef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:38.938329 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfda8f9-6960-4b2c-af75-77c14ab27ef5 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:38.948234 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-703c3625-4bf2-49c3-b49d-0a68c805c004 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:38.949661 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-703c3625-4bf2-49c3-b49d-0a68c805c004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:38.950225 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 799/1615] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:32:38 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:38.951118 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfda8f9-6960-4b2c-af75-77c14ab27ef5 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:38.960447 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 817/1616] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:38 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:38.961962 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfda8f9-6960-4b2c-af75-77c14ab27ef5 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:38.965026 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c3625-4bf2-49c3-b49d-0a68c805c004 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:38.975349 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28942fa-0fba-4774-a7c5-680d3940a13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:38.976035 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28942fa-0fba-4774-a7c5-680d3940a13d admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:38.976035 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28942fa-0fba-4774-a7c5-680d3940a13d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:38.976460 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28942fa-0fba-4774-a7c5-680d3940a13d admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:38.976939 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28942fa-0fba-4774-a7c5-680d3940a13d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:38.979381 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28942fa-0fba-4774-a7c5-680d3940a13d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:38.987641 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c3625-4bf2-49c3-b49d-0a68c805c004 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:38.994019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec3835-6cb6-4651-9896-971ea5a3d59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:38.995270 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec3835-6cb6-4651-9896-971ea5a3d59d admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:38.995785 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec3835-6cb6-4651-9896-971ea5a3d59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:38.996309 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec3835-6cb6-4651-9896-971ea5a3d59d admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:38.997900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec3835-6cb6-4651-9896-971ea5a3d59d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:39.001841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec3835-6cb6-4651-9896-971ea5a3d59d admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:39.023418 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 800/1617] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:32:38 2022] POST /identity/v3/projects => generated 382 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:39.024637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28942fa-0fba-4774-a7c5-680d3940a13d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:39.071368 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 818/1618] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:32:38 2022] POST /identity/v3/projects => generated 374 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:39.074761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec3835-6cb6-4651-9896-971ea5a3d59d admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:39.121295 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28942fa-0fba-4774-a7c5-680d3940a13d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:39.223790 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b16b23-9f88-48dd-81ed-5c2e260dcb71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:39.225151 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b16b23-9f88-48dd-81ed-5c2e260dcb71 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:39.225965 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b16b23-9f88-48dd-81ed-5c2e260dcb71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:39.226496 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b16b23-9f88-48dd-81ed-5c2e260dcb71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:39.261985 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec3835-6cb6-4651-9896-971ea5a3d59d admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:39.272144 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d953e4-506e-4087-94d8-9518db2244fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:39.273212 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d953e4-506e-4087-94d8-9518db2244fc service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:39.274244 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d953e4-506e-4087-94d8-9518db2244fc service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:39.274751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d953e4-506e-4087-94d8-9518db2244fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:39.295203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:39.354806 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b16b23-9f88-48dd-81ed-5c2e260dcb71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:39.357691 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b16b23-9f88-48dd-81ed-5c2e260dcb71 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:39.369322 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d953e4-506e-4087-94d8-9518db2244fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:39.371311 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d953e4-506e-4087-94d8-9518db2244fc service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:39.383686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 801/1619] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:32:39 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:39.387318 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb96579-445b-4da1-9967-63e94ab012e0 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:39.387706 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb96579-445b-4da1-9967-63e94ab012e0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:39.388097 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb96579-445b-4da1-9967-63e94ab012e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:39.389117 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 819/1620] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:32:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:39.390389 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d953e4-506e-4087-94d8-9518db2244fc service nova] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:39.415222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d953e4-506e-4087-94d8-9518db2244fc service nova] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:39.422694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581880b-5dfd-4455-aca2-31a8cc8508f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:39.423456 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581880b-5dfd-4455-aca2-31a8cc8508f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:39.423741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581880b-5dfd-4455-aca2-31a8cc8508f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:39.424056 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581880b-5dfd-4455-aca2-31a8cc8508f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:39.429028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7cb96579-445b-4da1-9967-63e94ab012e0 None None] MFA Rules not processed for user `0d947aeb749546309a8d4b7938087ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:39.430305 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7cb96579-445b-4da1-9967-63e94ab012e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:39.436519 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581880b-5dfd-4455-aca2-31a8cc8508f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:39.437986 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581880b-5dfd-4455-aca2-31a8cc8508f6 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:39.441085 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 802/1621] 10.222.0.121 () {58 vars in 1064 bytes} [Thu Aug 4 10:32:39 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:39.442404 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb96579-445b-4da1-9967-63e94ab012e0 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:39.456038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb96579-445b-4da1-9967-63e94ab012e0 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:39.461451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ccb00-d84f-402d-a23b-a7b6b0205163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:39.462127 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ccb00-d84f-402d-a23b-a7b6b0205163 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:39.462508 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ccb00-d84f-402d-a23b-a7b6b0205163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:39.463566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ccb00-d84f-402d-a23b-a7b6b0205163 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:39.464314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ccb00-d84f-402d-a23b-a7b6b0205163 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:39.466375 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ccb00-d84f-402d-a23b-a7b6b0205163 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:39.475693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 820/1622] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:32:39 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:39.478810 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581880b-5dfd-4455-aca2-31a8cc8508f6 service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:39.509562 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 803/1623] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:32:39 2022] POST /identity/v3/users => generated 298 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:39.511638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581880b-5dfd-4455-aca2-31a8cc8508f6 service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:39.513965 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ccb00-d84f-402d-a23b-a7b6b0205163 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:39.519098 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f06bc-816c-4d21-98a7-f4542b6e8559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:39.521894 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f06bc-816c-4d21-98a7-f4542b6e8559 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:39.522193 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f06bc-816c-4d21-98a7-f4542b6e8559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:39.522642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f06bc-816c-4d21-98a7-f4542b6e8559 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:39.524200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f06bc-816c-4d21-98a7-f4542b6e8559 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:39.528946 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f06bc-816c-4d21-98a7-f4542b6e8559 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:39.535870 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ccb00-d84f-402d-a23b-a7b6b0205163 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:39.544836 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c044b-912b-46f9-890b-fc852530bdf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:39.545811 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c044b-912b-46f9-890b-fc852530bdf0 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:39.546179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c044b-912b-46f9-890b-fc852530bdf0 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:39.546508 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c044b-912b-46f9-890b-fc852530bdf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:39.580781 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 821/1624] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:32:39 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:39.583015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f06bc-816c-4d21-98a7-f4542b6e8559 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:39.611932 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f06bc-816c-4d21-98a7-f4542b6e8559 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:39.623233 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4da95-f6c6-44e8-a39c-077d6cea64e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:39.625609 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4da95-f6c6-44e8-a39c-077d6cea64e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:39.626541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4da95-f6c6-44e8-a39c-077d6cea64e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:39.627215 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4da95-f6c6-44e8-a39c-077d6cea64e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:39.630141 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e4da95-f6c6-44e8-a39c-077d6cea64e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:39.634277 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e4da95-f6c6-44e8-a39c-077d6cea64e2 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:39.660954 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c044b-912b-46f9-890b-fc852530bdf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:39.662386 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 822/1625] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:39 2022] GET /identity/v3/roles => generated 1427 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:39.665487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4da95-f6c6-44e8-a39c-077d6cea64e2 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:39.666857 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c044b-912b-46f9-890b-fc852530bdf0 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:39.680422 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 804/1626] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:32:39 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:39.681405 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c044b-912b-46f9-890b-fc852530bdf0 service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:39.696706 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c044b-912b-46f9-890b-fc852530bdf0 service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:39.801146 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9603c6-76a0-49cb-8899-007edffd2801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:39.801146 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9603c6-76a0-49cb-8899-007edffd2801 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:39.801146 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9603c6-76a0-49cb-8899-007edffd2801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:39.801146 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9603c6-76a0-49cb-8899-007edffd2801 admin admin] PATH_INFO: `/v3/projects/64e299419ec843c3be8a8e2d061f2ec2/users/f969009b2885414da735310a4d844b92/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:39.802385 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4da95-f6c6-44e8-a39c-077d6cea64e2 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:39.802385 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146564f1-d803-421c-aa44-237b6ad8a024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:39.802385 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146564f1-d803-421c-aa44-237b6ad8a024 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:39.802385 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146564f1-d803-421c-aa44-237b6ad8a024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:39.802868 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146564f1-d803-421c-aa44-237b6ad8a024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:39.802868 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146564f1-d803-421c-aa44-237b6ad8a024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:39.802868 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146564f1-d803-421c-aa44-237b6ad8a024 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:39.803536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9603c6-76a0-49cb-8899-007edffd2801 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64e299419ec843c3be8a8e2d061f2ec2, user_id=f969009b2885414da735310a4d844b92, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:39.807595 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9603c6-76a0-49cb-8899-007edffd2801 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:39.835700 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 805/1627] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:32:39 2022] PUT /identity/v3/projects/64e299419ec843c3be8a8e2d061f2ec2/users/f969009b2885414da735310a4d844b92/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:39.842110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9603c6-76a0-49cb-8899-007edffd2801 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:39.867759 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 823/1628] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:39 2022] GET /identity/v3/roles => generated 1427 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:39.868685 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146564f1-d803-421c-aa44-237b6ad8a024 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:39.900855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146564f1-d803-421c-aa44-237b6ad8a024 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:39.910059 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:40.032291 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9603c6-76a0-49cb-8899-007edffd2801 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:40.039868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a05ae3-8048-4493-97bb-8288f57d75f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:40.040760 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a05ae3-8048-4493-97bb-8288f57d75f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:40.041207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a05ae3-8048-4493-97bb-8288f57d75f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:40.041399 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a05ae3-8048-4493-97bb-8288f57d75f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:40.042084 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a05ae3-8048-4493-97bb-8288f57d75f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:40.043890 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a05ae3-8048-4493-97bb-8288f57d75f7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:40.059672 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 806/1629] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:39 2022] GET /identity/v3/roles => generated 1427 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:40.061473 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68882c6f-d802-474f-b2aa-95d4dcfa87c6 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:40.061777 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68882c6f-d802-474f-b2aa-95d4dcfa87c6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:40.062173 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68882c6f-d802-474f-b2aa-95d4dcfa87c6 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:40.070467 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 807/1630] 10.222.0.121 () {54 vars in 917 bytes} [Thu Aug 4 10:32:40 2022] GET /identity => generated 271 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:32:40.071408 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68882c6f-d802-474f-b2aa-95d4dcfa87c6 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:40.088698 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68882c6f-d802-474f-b2aa-95d4dcfa87c6 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:40.094038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21aec4-eb37-4e27-8e30-28f3e743a32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:40.094993 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21aec4-eb37-4e27-8e30-28f3e743a32e admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:40.095280 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21aec4-eb37-4e27-8e30-28f3e743a32e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:40.095558 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21aec4-eb37-4e27-8e30-28f3e743a32e admin admin] PATH_INFO: `/v3/projects/64e299419ec843c3be8a8e2d061f2ec2/users/f969009b2885414da735310a4d844b92/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:40.098905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21aec4-eb37-4e27-8e30-28f3e743a32e admin admin] RBAC: Authorizing `identity:create_grant(project_id=64e299419ec843c3be8a8e2d061f2ec2, user_id=f969009b2885414da735310a4d844b92, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:40.118661 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c93644-b615-4280-8534-36a40446d049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:40.119813 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c93644-b615-4280-8534-36a40446d049 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:40.120524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c93644-b615-4280-8534-36a40446d049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:40.121135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c93644-b615-4280-8534-36a40446d049 admin admin] PATH_INFO: `/v3/projects/7cec3109bb454e3ca7aa6362d721321d/users/99364a3d9ad14a00ba90ba8ddc7dfa20/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:40.128507 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21aec4-eb37-4e27-8e30-28f3e743a32e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:40.140943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c93644-b615-4280-8534-36a40446d049 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cec3109bb454e3ca7aa6362d721321d, user_id=99364a3d9ad14a00ba90ba8ddc7dfa20, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:40.145980 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c93644-b615-4280-8534-36a40446d049 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:40.153727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 808/1631] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:32:40 2022] PUT /identity/v3/projects/64e299419ec843c3be8a8e2d061f2ec2/users/f969009b2885414da735310a4d844b92/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:40.155101 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4cbc5-18f5-4506-8c57-73c0d9b80be7 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:40.155530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4cbc5-18f5-4506-8c57-73c0d9b80be7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:40.155665 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4cbc5-18f5-4506-8c57-73c0d9b80be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:40.182188 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:40.192489 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 824/1632] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:32:39 2022] PUT /identity/v3/projects/7cec3109bb454e3ca7aa6362d721321d/users/99364a3d9ad14a00ba90ba8ddc7dfa20/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:40.196669 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e056ee-c932-4408-97a7-903f9dd66724 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:40.197072 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e056ee-c932-4408-97a7-903f9dd66724 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:40.197237 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e056ee-c932-4408-97a7-903f9dd66724 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:40.198894 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 825/1633] 10.222.0.121 () {54 vars in 917 bytes} [Thu Aug 4 10:32:40 2022] GET /identity => generated 271 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:32:40.206543 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da2572-07be-4881-92dd-1a6a9ccae332 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:40.206673 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da2572-07be-4881-92dd-1a6a9ccae332 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:40.206996 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da2572-07be-4881-92dd-1a6a9ccae332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:40.214264 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-99d4cbc5-18f5-4506-8c57-73c0d9b80be7 None None] MFA Rules not processed for user `0d947aeb749546309a8d4b7938087ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:40.215661 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-99d4cbc5-18f5-4506-8c57-73c0d9b80be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:40.269612 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-55da2572-07be-4881-92dd-1a6a9ccae332 None None] MFA Rules not processed for user `f969009b2885414da735310a4d844b92`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:40.271220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-55da2572-07be-4881-92dd-1a6a9ccae332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:40.334334 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 809/1634] 10.222.0.121 () {58 vars in 1014 bytes} [Thu Aug 4 10:32:40 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:40.335251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4cbc5-18f5-4506-8c57-73c0d9b80be7 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:40.456585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4cbc5-18f5-4506-8c57-73c0d9b80be7 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:40.460904 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 826/1635] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:40 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:40.465233 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b1da4-8087-4b1e-99fe-077e67fe2c7b None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:40.465931 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b1da4-8087-4b1e-99fe-077e67fe2c7b None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:40.465931 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b1da4-8087-4b1e-99fe-077e67fe2c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:40.473054 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f438b0-1241-4e18-a3fa-e0cfe61d6980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:40.473923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f438b0-1241-4e18-a3fa-e0cfe61d6980 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:40.474044 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f438b0-1241-4e18-a3fa-e0cfe61d6980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:40.474783 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f438b0-1241-4e18-a3fa-e0cfe61d6980 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:40.475333 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f438b0-1241-4e18-a3fa-e0cfe61d6980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:40.477091 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f438b0-1241-4e18-a3fa-e0cfe61d6980 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:40.510667 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 810/1636] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:40 2022] GET /identity/v3/roles => generated 1427 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:40.511854 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f438b0-1241-4e18-a3fa-e0cfe61d6980 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:40.521331 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-322b1da4-8087-4b1e-99fe-077e67fe2c7b None None] MFA Rules not processed for user `0d947aeb749546309a8d4b7938087ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:40.522901 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-322b1da4-8087-4b1e-99fe-077e67fe2c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:40.545633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 827/1637] 10.222.0.121 () {58 vars in 1014 bytes} [Thu Aug 4 10:32:40 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:40.546970 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b1da4-8087-4b1e-99fe-077e67fe2c7b None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:40.644332 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f438b0-1241-4e18-a3fa-e0cfe61d6980 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:40.653898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf89ea-3ab5-402b-8b13-4c5e089009cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:40.654732 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf89ea-3ab5-402b-8b13-4c5e089009cb service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:40.655021 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf89ea-3ab5-402b-8b13-4c5e089009cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:40.655300 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf89ea-3ab5-402b-8b13-4c5e089009cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:40.697114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bf89ea-3ab5-402b-8b13-4c5e089009cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:40.698009 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b1da4-8087-4b1e-99fe-077e67fe2c7b None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:40.698624 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bf89ea-3ab5-402b-8b13-4c5e089009cb service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:40.703752 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f20a31-be45-4f3f-b344-061144c22633 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:40.705247 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f20a31-be45-4f3f-b344-061144c22633 service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:40.705858 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f20a31-be45-4f3f-b344-061144c22633 service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:40.706338 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f20a31-be45-4f3f-b344-061144c22633 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:40.725019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f20a31-be45-4f3f-b344-061144c22633 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:40.728418 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 811/1638] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:32:40 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:40.729358 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf89ea-3ab5-402b-8b13-4c5e089009cb service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:40.755106 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f20a31-be45-4f3f-b344-061144c22633 service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:40.757196 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf89ea-3ab5-402b-8b13-4c5e089009cb service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:40.768284 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03c15c-df35-499a-b1b5-1c100c598dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:40.769020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03c15c-df35-499a-b1b5-1c100c598dab admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:40.769314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03c15c-df35-499a-b1b5-1c100c598dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:40.769624 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03c15c-df35-499a-b1b5-1c100c598dab admin admin] PATH_INFO: `/v3/projects/7cec3109bb454e3ca7aa6362d721321d/users/99364a3d9ad14a00ba90ba8ddc7dfa20/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:40.785768 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b03c15c-df35-499a-b1b5-1c100c598dab admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cec3109bb454e3ca7aa6362d721321d, user_id=99364a3d9ad14a00ba90ba8ddc7dfa20, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:40.792962 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b03c15c-df35-499a-b1b5-1c100c598dab admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:40.804454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 828/1639] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:32:40 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:40.848208 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 812/1640] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:32:40 2022] PUT /identity/v3/projects/7cec3109bb454e3ca7aa6362d721321d/users/99364a3d9ad14a00ba90ba8ddc7dfa20/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:40.869089 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6024f3-3ea7-43f1-8c90-93efd3f961b4 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:40.870020 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6024f3-3ea7-43f1-8c90-93efd3f961b4 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:40.871811 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6024f3-3ea7-43f1-8c90-93efd3f961b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:40.917290 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03c15c-df35-499a-b1b5-1c100c598dab admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:40.940375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:40.975015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-3e6024f3-3ea7-43f1-8c90-93efd3f961b4 None None] MFA Rules not processed for user `99364a3d9ad14a00ba90ba8ddc7dfa20`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:40.977992 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-3e6024f3-3ea7-43f1-8c90-93efd3f961b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:40.997194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:41.120985 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03c15c-df35-499a-b1b5-1c100c598dab admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:41.125820 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4b263-cec7-438d-8330-65374688dfdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:41.126732 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4b263-cec7-438d-8330-65374688dfdb service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:41.127188 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4b263-cec7-438d-8330-65374688dfdb service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:41.129446 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4b263-cec7-438d-8330-65374688dfdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:41.221794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 829/1641] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:40 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:32:41.340827 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c4b263-cec7-438d-8330-65374688dfdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:41.343178 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c4b263-cec7-438d-8330-65374688dfdb service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:41.377963 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 813/1642] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:32:40 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:41.424777 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6024f3-3ea7-43f1-8c90-93efd3f961b4 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:41.433646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4b263-cec7-438d-8330-65374688dfdb service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:41.452423 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4b263-cec7-438d-8330-65374688dfdb service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:41.472399 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c4ef3-5de0-4148-8686-ef99a9a603c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:41.473447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c4ef3-5de0-4148-8686-ef99a9a603c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:41.474880 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c4ef3-5de0-4148-8686-ef99a9a603c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:41.475170 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c4ef3-5de0-4148-8686-ef99a9a603c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:41.504939 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c4ef3-5de0-4148-8686-ef99a9a603c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:41.510086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c4ef3-5de0-4148-8686-ef99a9a603c3 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:41.564453 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 814/1643] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:32:41 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:32:41.662723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6024f3-3ea7-43f1-8c90-93efd3f961b4 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:41.675864 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f257a52-6007-4c13-8f01-856f400d2408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:41.676791 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f257a52-6007-4c13-8f01-856f400d2408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:41.677271 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f257a52-6007-4c13-8f01-856f400d2408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:41.677654 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f257a52-6007-4c13-8f01-856f400d2408 admin admin] PATH_INFO: `/v3/users/7631e70ced3b4e72b64d971d644fcab2` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:41.680873 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f257a52-6007-4c13-8f01-856f400d2408 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7631e70ced3b4e72b64d971d644fcab2)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:41.683664 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f257a52-6007-4c13-8f01-856f400d2408 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:41.826061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-9f257a52-6007-4c13-8f01-856f400d2408 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7631e70ced3b4e72b64d971d644fcab2'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:41.827119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-9f257a52-6007-4c13-8f01-856f400d2408 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7631e70ced3b4e72b64d971d644fcab2'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:41.856501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-9f257a52-6007-4c13-8f01-856f400d2408 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7631e70ced3b4e72b64d971d644fcab2'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:41.896887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-9f257a52-6007-4c13-8f01-856f400d2408 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7631e70ced3b4e72b64d971d644fcab2'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:41.930167 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-9f257a52-6007-4c13-8f01-856f400d2408 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7631e70ced3b4e72b64d971d644fcab2'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:41.945982 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 830/1644] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:32:41 2022] DELETE /identity/v3/users/7631e70ced3b4e72b64d971d644fcab2 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:41.949919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e42b4e-0934-4cea-9ab1-733cc7ab6134 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:41.950456 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e42b4e-0934-4cea-9ab1-733cc7ab6134 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:41.950550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e42b4e-0934-4cea-9ab1-733cc7ab6134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:42.006293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b3e42b4e-0934-4cea-9ab1-733cc7ab6134 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:42.007041 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b3e42b4e-0934-4cea-9ab1-733cc7ab6134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:42.205391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 815/1645] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:42.284673 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f257a52-6007-4c13-8f01-856f400d2408 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:42.287652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-9f257a52-6007-4c13-8f01-856f400d2408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:42.377364 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:42.527546 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f257a52-6007-4c13-8f01-856f400d2408 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:42.547158 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024baf2-e4af-4003-a75c-9e885237549d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:42.548305 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024baf2-e4af-4003-a75c-9e885237549d service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:42.548957 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024baf2-e4af-4003-a75c-9e885237549d service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:42.549179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024baf2-e4af-4003-a75c-9e885237549d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:42.631441 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024baf2-e4af-4003-a75c-9e885237549d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:42.631441 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024baf2-e4af-4003-a75c-9e885237549d service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:42.631441 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 831/1646] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:32:42 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:42.951308 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e42b4e-0934-4cea-9ab1-733cc7ab6134 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:42.953828 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b3e42b4e-0934-4cea-9ab1-733cc7ab6134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:42.978979 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e42b4e-0934-4cea-9ab1-733cc7ab6134 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:42.988689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663176e-cfb1-43c8-a210-1455fe859873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:42.989379 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663176e-cfb1-43c8-a210-1455fe859873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:42.989674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663176e-cfb1-43c8-a210-1455fe859873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:42.989959 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663176e-cfb1-43c8-a210-1455fe859873 admin admin] PATH_INFO: `/v3/users/e8fc011e92854d219da5f0f8a5fba798` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:42.993634 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c663176e-cfb1-43c8-a210-1455fe859873 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8fc011e92854d219da5f0f8a5fba798)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:42.994854 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: WARNING py.warnings [None req-c663176e-cfb1-43c8-a210-1455fe859873 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:32:42.994854 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: warnings.warn(msg) Aug 04 10:32:42.994854 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: Aug 04 10:32:42.996648 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c663176e-cfb1-43c8-a210-1455fe859873 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:43.009982 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024baf2-e4af-4003-a75c-9e885237549d service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:43.011552 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-4024baf2-e4af-4003-a75c-9e885237549d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:43.030107 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024baf2-e4af-4003-a75c-9e885237549d service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:43.036257 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d062e-54d2-48fe-bfd3-7dc0ec3097af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:43.038089 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d062e-54d2-48fe-bfd3-7dc0ec3097af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:43.038616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d062e-54d2-48fe-bfd3-7dc0ec3097af admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:43.039127 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d062e-54d2-48fe-bfd3-7dc0ec3097af admin admin] PATH_INFO: `/v3/users/4ec1e2d4753d47d2a565485c65948746` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:43.040750 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d062e-54d2-48fe-bfd3-7dc0ec3097af admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ec1e2d4753d47d2a565485c65948746)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:43.043122 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d062e-54d2-48fe-bfd3-7dc0ec3097af admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:43.140626 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-187d062e-54d2-48fe-bfd3-7dc0ec3097af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ec1e2d4753d47d2a565485c65948746'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:43.142761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-187d062e-54d2-48fe-bfd3-7dc0ec3097af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ec1e2d4753d47d2a565485c65948746'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:43.146896 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:43.156735 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c663176e-cfb1-43c8-a210-1455fe859873 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8fc011e92854d219da5f0f8a5fba798'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:43.181230 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c663176e-cfb1-43c8-a210-1455fe859873 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8fc011e92854d219da5f0f8a5fba798'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:43.185691 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c663176e-cfb1-43c8-a210-1455fe859873 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8fc011e92854d219da5f0f8a5fba798'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:43.190391 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-187d062e-54d2-48fe-bfd3-7dc0ec3097af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ec1e2d4753d47d2a565485c65948746'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:43.206709 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-187d062e-54d2-48fe-bfd3-7dc0ec3097af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ec1e2d4753d47d2a565485c65948746'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:43.220649 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-187d062e-54d2-48fe-bfd3-7dc0ec3097af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ec1e2d4753d47d2a565485c65948746'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:43.229944 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c663176e-cfb1-43c8-a210-1455fe859873 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8fc011e92854d219da5f0f8a5fba798'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:43.232055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 832/1647] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:32:43 2022] DELETE /identity/v3/users/4ec1e2d4753d47d2a565485c65948746 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:32:43.237509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c663176e-cfb1-43c8-a210-1455fe859873 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8fc011e92854d219da5f0f8a5fba798'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:43.238233 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fedda-e501-4540-859e-67acd369a629 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:43.239333 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fedda-e501-4540-859e-67acd369a629 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:43.240089 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fedda-e501-4540-859e-67acd369a629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:43.256705 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 816/1648] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:32:42 2022] DELETE /identity/v3/users/e8fc011e92854d219da5f0f8a5fba798 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:43.271022 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef18978-4962-4f4d-99be-83e222de3065 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:43.271524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef18978-4962-4f4d-99be-83e222de3065 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:43.271929 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef18978-4962-4f4d-99be-83e222de3065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:43.289067 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-0d9fedda-e501-4540-859e-67acd369a629 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:43.295657 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-0d9fedda-e501-4540-859e-67acd369a629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:43.345566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-eef18978-4962-4f4d-99be-83e222de3065 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:43.355640 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-eef18978-4962-4f4d-99be-83e222de3065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:43.484354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 833/1649] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:43.534888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 817/1650] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:43.539096 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fedda-e501-4540-859e-67acd369a629 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:43.541487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-0d9fedda-e501-4540-859e-67acd369a629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:43.619391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef18978-4962-4f4d-99be-83e222de3065 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:43.620875 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-eef18978-4962-4f4d-99be-83e222de3065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:43.670325 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:43.736133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fedda-e501-4540-859e-67acd369a629 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:43.759679 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9631e7-3583-49f9-b497-373d2fc8bed1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:43.761126 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9631e7-3583-49f9-b497-373d2fc8bed1 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:43.765364 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9631e7-3583-49f9-b497-373d2fc8bed1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:43.766736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9631e7-3583-49f9-b497-373d2fc8bed1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:43.784172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef18978-4962-4f4d-99be-83e222de3065 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:43.790067 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9631e7-3583-49f9-b497-373d2fc8bed1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:43.793034 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9631e7-3583-49f9-b497-373d2fc8bed1 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:43.793996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5d56d-e1c6-4458-9ec1-95d9ba271788 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:43.795915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5d56d-e1c6-4458-9ec1-95d9ba271788 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:43.795915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5d56d-e1c6-4458-9ec1-95d9ba271788 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:43.796310 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5d56d-e1c6-4458-9ec1-95d9ba271788 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:43.821094 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5d56d-e1c6-4458-9ec1-95d9ba271788 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:43.822651 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5d56d-e1c6-4458-9ec1-95d9ba271788 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:43.839158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 818/1651] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:32:43 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:43.843309 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 834/1652] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:32:43 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:45.864780 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5d56d-e1c6-4458-9ec1-95d9ba271788 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:45.866859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-80c5d56d-e1c6-4458-9ec1-95d9ba271788 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:45.875188 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9631e7-3583-49f9-b497-373d2fc8bed1 service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:45.877539 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-5e9631e7-3583-49f9-b497-373d2fc8bed1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:45.898396 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9631e7-3583-49f9-b497-373d2fc8bed1 service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:45.915016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5d56d-e1c6-4458-9ec1-95d9ba271788 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:45.917254 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34288dd-06df-433b-b280-0b73cdcfa1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:45.917459 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34288dd-06df-433b-b280-0b73cdcfa1cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:45.917715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34288dd-06df-433b-b280-0b73cdcfa1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:45.918004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34288dd-06df-433b-b280-0b73cdcfa1cb admin admin] PATH_INFO: `/v3/projects/c1dbefa7237d4118a80cac6078fd2023` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:45.921755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34288dd-06df-433b-b280-0b73cdcfa1cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1dbefa7237d4118a80cac6078fd2023)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:45.924432 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: WARNING py.warnings [None req-e34288dd-06df-433b-b280-0b73cdcfa1cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 10:32:45.924432 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: warnings.warn(msg) Aug 04 10:32:45.924432 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: Aug 04 10:32:45.925664 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34288dd-06df-433b-b280-0b73cdcfa1cb admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:45.929356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5787cc24-8c67-48f0-97f0-2319e5d27796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:45.930375 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787cc24-8c67-48f0-97f0-2319e5d27796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:45.931442 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787cc24-8c67-48f0-97f0-2319e5d27796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:45.932295 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787cc24-8c67-48f0-97f0-2319e5d27796 admin admin] PATH_INFO: `/v3/projects/e2c1805521bc43f8a663bc01eeddda2f` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:45.939234 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5787cc24-8c67-48f0-97f0-2319e5d27796 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2c1805521bc43f8a663bc01eeddda2f)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:45.944135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5787cc24-8c67-48f0-97f0-2319e5d27796 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:46.097835 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e34288dd-06df-433b-b280-0b73cdcfa1cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1dbefa7237d4118a80cac6078fd2023', 'request_id': 'req-e34288dd-06df-433b-b280-0b73cdcfa1cb'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:46.125218 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5787cc24-8c67-48f0-97f0-2319e5d27796 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2c1805521bc43f8a663bc01eeddda2f', 'request_id': 'req-5787cc24-8c67-48f0-97f0-2319e5d27796'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:46.132080 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e34288dd-06df-433b-b280-0b73cdcfa1cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1dbefa7237d4118a80cac6078fd2023', 'request_id': 'req-e34288dd-06df-433b-b280-0b73cdcfa1cb'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:46.142954 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5787cc24-8c67-48f0-97f0-2319e5d27796 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2c1805521bc43f8a663bc01eeddda2f', 'request_id': 'req-5787cc24-8c67-48f0-97f0-2319e5d27796'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:46.150443 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e34288dd-06df-433b-b280-0b73cdcfa1cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1dbefa7237d4118a80cac6078fd2023', 'request_id': 'req-e34288dd-06df-433b-b280-0b73cdcfa1cb'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:46.154677 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:46.181395 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5787cc24-8c67-48f0-97f0-2319e5d27796 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2c1805521bc43f8a663bc01eeddda2f', 'request_id': 'req-5787cc24-8c67-48f0-97f0-2319e5d27796'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:46.230902 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5787cc24-8c67-48f0-97f0-2319e5d27796 admin admin] The token cache is being invalidate because project e2c1805521bc43f8a663bc01eeddda2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:46.230902 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5787cc24-8c67-48f0-97f0-2319e5d27796 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:46.232315 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5787cc24-8c67-48f0-97f0-2319e5d27796 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:46.234767 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e34288dd-06df-433b-b280-0b73cdcfa1cb admin admin] The token cache is being invalidate because project c1dbefa7237d4118a80cac6078fd2023 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:46.235889 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e34288dd-06df-433b-b280-0b73cdcfa1cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:46.238608 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e34288dd-06df-433b-b280-0b73cdcfa1cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:46.242006 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 819/1653] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:32:45 2022] DELETE /identity/v3/projects/e2c1805521bc43f8a663bc01eeddda2f => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:46.248457 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 835/1654] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:32:45 2022] DELETE /identity/v3/projects/c1dbefa7237d4118a80cac6078fd2023 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:46.255121 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34288dd-06df-433b-b280-0b73cdcfa1cb admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:46.255369 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5787cc24-8c67-48f0-97f0-2319e5d27796 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:46.257106 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5787cc24-8c67-48f0-97f0-2319e5d27796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:46.257344 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-e34288dd-06df-433b-b280-0b73cdcfa1cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:46.440539 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34288dd-06df-433b-b280-0b73cdcfa1cb admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:46.465863 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5787cc24-8c67-48f0-97f0-2319e5d27796 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:46.473723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f01be-d92f-46fb-9775-eeac6ba174f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:46.474844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f01be-d92f-46fb-9775-eeac6ba174f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:46.475845 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f01be-d92f-46fb-9775-eeac6ba174f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:46.476594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f01be-d92f-46fb-9775-eeac6ba174f1 admin admin] PATH_INFO: `/v3/users/f969009b2885414da735310a4d844b92` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:46.487701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f01be-d92f-46fb-9775-eeac6ba174f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f969009b2885414da735310a4d844b92)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:46.498591 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f01be-d92f-46fb-9775-eeac6ba174f1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:46.630684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d34f01be-d92f-46fb-9775-eeac6ba174f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f969009b2885414da735310a4d844b92'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:46.647779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d34f01be-d92f-46fb-9775-eeac6ba174f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f969009b2885414da735310a4d844b92'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:46.649365 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d34f01be-d92f-46fb-9775-eeac6ba174f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f969009b2885414da735310a4d844b92'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:46.661412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421d354-1b3e-4324-baf2-a0d43e4d8536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:46.662125 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421d354-1b3e-4324-baf2-a0d43e4d8536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:46.662406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421d354-1b3e-4324-baf2-a0d43e4d8536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:46.662685 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421d354-1b3e-4324-baf2-a0d43e4d8536 admin admin] PATH_INFO: `/v3/users/99364a3d9ad14a00ba90ba8ddc7dfa20` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:46.667551 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421d354-1b3e-4324-baf2-a0d43e4d8536 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99364a3d9ad14a00ba90ba8ddc7dfa20)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:46.673982 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421d354-1b3e-4324-baf2-a0d43e4d8536 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:46.682840 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d34f01be-d92f-46fb-9775-eeac6ba174f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f969009b2885414da735310a4d844b92'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:46.683586 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d34f01be-d92f-46fb-9775-eeac6ba174f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f969009b2885414da735310a4d844b92'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:46.700328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:46.725703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 820/1655] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:32:46 2022] DELETE /identity/v3/users/f969009b2885414da735310a4d844b92 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:46.726948 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f01be-d92f-46fb-9775-eeac6ba174f1 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:46.731195 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d34f01be-d92f-46fb-9775-eeac6ba174f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:46.856896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6421d354-1b3e-4324-baf2-a0d43e4d8536 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99364a3d9ad14a00ba90ba8ddc7dfa20'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:46.861678 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6421d354-1b3e-4324-baf2-a0d43e4d8536 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99364a3d9ad14a00ba90ba8ddc7dfa20'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:46.912743 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6421d354-1b3e-4324-baf2-a0d43e4d8536 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99364a3d9ad14a00ba90ba8ddc7dfa20'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:46.920975 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f01be-d92f-46fb-9775-eeac6ba174f1 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:46.926728 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6421d354-1b3e-4324-baf2-a0d43e4d8536 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99364a3d9ad14a00ba90ba8ddc7dfa20'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:46.929794 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c3a1d-807c-4c76-8460-a708bc515343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:46.930841 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c3a1d-807c-4c76-8460-a708bc515343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:46.931278 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c3a1d-807c-4c76-8460-a708bc515343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:46.931680 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c3a1d-807c-4c76-8460-a708bc515343 admin admin] PATH_INFO: `/v3/projects/5d18be11fe1d499dac0af5ee521d0203` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:46.935384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c3a1d-807c-4c76-8460-a708bc515343 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d18be11fe1d499dac0af5ee521d0203)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:46.939025 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c3a1d-807c-4c76-8460-a708bc515343 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:46.953368 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6421d354-1b3e-4324-baf2-a0d43e4d8536 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99364a3d9ad14a00ba90ba8ddc7dfa20'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:46.970600 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 836/1656] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:32:46 2022] DELETE /identity/v3/users/99364a3d9ad14a00ba90ba8ddc7dfa20 => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:46.978728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:47.063330 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4b3c3a1d-807c-4c76-8460-a708bc515343 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d18be11fe1d499dac0af5ee521d0203', 'request_id': 'req-4b3c3a1d-807c-4c76-8460-a708bc515343'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:47.076030 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4b3c3a1d-807c-4c76-8460-a708bc515343 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d18be11fe1d499dac0af5ee521d0203', 'request_id': 'req-4b3c3a1d-807c-4c76-8460-a708bc515343'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:47.088220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4b3c3a1d-807c-4c76-8460-a708bc515343 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d18be11fe1d499dac0af5ee521d0203', 'request_id': 'req-4b3c3a1d-807c-4c76-8460-a708bc515343'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:47.122336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4b3c3a1d-807c-4c76-8460-a708bc515343 admin admin] The token cache is being invalidate because project 5d18be11fe1d499dac0af5ee521d0203 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:47.125824 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4b3c3a1d-807c-4c76-8460-a708bc515343 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:47.129221 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4b3c3a1d-807c-4c76-8460-a708bc515343 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:47.143273 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 821/1657] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:32:46 2022] DELETE /identity/v3/projects/5d18be11fe1d499dac0af5ee521d0203 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:47.145862 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfb239-f660-46e9-86de-c775bc704188 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:47.146354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfb239-f660-46e9-86de-c775bc704188 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:47.146848 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfb239-f660-46e9-86de-c775bc704188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:47.213561 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-9adfb239-f660-46e9-86de-c775bc704188 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:47.215398 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-9adfb239-f660-46e9-86de-c775bc704188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:47.359819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 837/1658] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:47.360365 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2357bd-5260-40e0-ab65-aa3869b4b0e7 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:47.360627 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2357bd-5260-40e0-ab65-aa3869b4b0e7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:47.361075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2357bd-5260-40e0-ab65-aa3869b4b0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:47.396201 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-3b2357bd-5260-40e0-ab65-aa3869b4b0e7 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:47.397625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3b2357bd-5260-40e0-ab65-aa3869b4b0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:47.408392 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 822/1659] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:47.412981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfb239-f660-46e9-86de-c775bc704188 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:47.425864 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfb239-f660-46e9-86de-c775bc704188 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:47.439362 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9b5a6-8ccc-480c-96ab-46e37a03ff1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:47.440419 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9b5a6-8ccc-480c-96ab-46e37a03ff1a admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:47.440804 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9b5a6-8ccc-480c-96ab-46e37a03ff1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:47.441256 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9b5a6-8ccc-480c-96ab-46e37a03ff1a admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:47.445608 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9b5a6-8ccc-480c-96ab-46e37a03ff1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:32:47.446161 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9b5a6-8ccc-480c-96ab-46e37a03ff1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:47.450139 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9b5a6-8ccc-480c-96ab-46e37a03ff1a admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:47.493755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 838/1660] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:32:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:32:47.499939 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2357bd-5260-40e0-ab65-aa3869b4b0e7 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:47.517684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2357bd-5260-40e0-ab65-aa3869b4b0e7 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:47.530092 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc24794-206f-4714-9dde-d9e13ccb1f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:47.530843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc24794-206f-4714-9dde-d9e13ccb1f2b admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:47.531248 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc24794-206f-4714-9dde-d9e13ccb1f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:47.531530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc24794-206f-4714-9dde-d9e13ccb1f2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:47.532450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc24794-206f-4714-9dde-d9e13ccb1f2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:47.539037 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc24794-206f-4714-9dde-d9e13ccb1f2b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:47.597647 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 823/1661] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:32:47 2022] POST /identity/v3/projects => generated 370 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:47.602458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9b5a6-8ccc-480c-96ab-46e37a03ff1a admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:47.621759 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9b5a6-8ccc-480c-96ab-46e37a03ff1a admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:47.811336 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f2ad8-0d57-4c91-82eb-1ef7538a290a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:47.816759 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f2ad8-0d57-4c91-82eb-1ef7538a290a admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:47.819272 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f2ad8-0d57-4c91-82eb-1ef7538a290a admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:47.823284 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f2ad8-0d57-4c91-82eb-1ef7538a290a admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:47.829685 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f2ad8-0d57-4c91-82eb-1ef7538a290a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:47.843648 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f2ad8-0d57-4c91-82eb-1ef7538a290a admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:47.908614 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:47.921310 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc24794-206f-4714-9dde-d9e13ccb1f2b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:47.922985 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 839/1662] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:32:47 2022] POST /identity/v3/users => generated 286 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:47.934628 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc24794-206f-4714-9dde-d9e13ccb1f2b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:47.950567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c95ca7-ab38-4f1a-92bd-c7293bdc4bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:47.961301 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c95ca7-ab38-4f1a-92bd-c7293bdc4bba admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:47.962199 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c95ca7-ab38-4f1a-92bd-c7293bdc4bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:47.962902 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c95ca7-ab38-4f1a-92bd-c7293bdc4bba admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:47.965292 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c95ca7-ab38-4f1a-92bd-c7293bdc4bba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:47.967998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c95ca7-ab38-4f1a-92bd-c7293bdc4bba admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:47.999836 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f2ad8-0d57-4c91-82eb-1ef7538a290a admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:48.001055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 824/1663] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:47 2022] GET /identity/v3/roles => generated 1427 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:48.016826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f2ad8-0d57-4c91-82eb-1ef7538a290a admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:48.043349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79237b2-23d6-40c2-af49-93a441622608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:48.044124 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79237b2-23d6-40c2-af49-93a441622608 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:48.044571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79237b2-23d6-40c2-af49-93a441622608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:48.045844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79237b2-23d6-40c2-af49-93a441622608 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:48.046948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79237b2-23d6-40c2-af49-93a441622608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:48.054629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79237b2-23d6-40c2-af49-93a441622608 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:48.128994 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 840/1664] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:47 2022] GET /identity/v3/roles => generated 1427 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:32:48.134651 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c95ca7-ab38-4f1a-92bd-c7293bdc4bba admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:48.153965 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c95ca7-ab38-4f1a-92bd-c7293bdc4bba admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:48.160178 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695f3e7-a0b5-4f62-9c28-39553677178f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:48.160936 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695f3e7-a0b5-4f62-9c28-39553677178f admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:48.161316 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695f3e7-a0b5-4f62-9c28-39553677178f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:48.161608 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695f3e7-a0b5-4f62-9c28-39553677178f admin admin] PATH_INFO: `/v3/projects/dc5cb824051a46cea9fa28748570cf8d/users/98ab41698d2841aeafafc2b4d0ab8d26/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:48.224955 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695f3e7-a0b5-4f62-9c28-39553677178f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc5cb824051a46cea9fa28748570cf8d, user_id=98ab41698d2841aeafafc2b4d0ab8d26, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:48.231039 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695f3e7-a0b5-4f62-9c28-39553677178f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:48.265335 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 825/1665] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:32:48 2022] PUT /identity/v3/projects/dc5cb824051a46cea9fa28748570cf8d/users/98ab41698d2841aeafafc2b4d0ab8d26/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:48.277553 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339de55-bb8a-428d-9e72-6ef184d16b5b None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:48.277850 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339de55-bb8a-428d-9e72-6ef184d16b5b None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:48.278134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339de55-bb8a-428d-9e72-6ef184d16b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:48.319650 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-0339de55-bb8a-428d-9e72-6ef184d16b5b None None] MFA Rules not processed for user `98ab41698d2841aeafafc2b4d0ab8d26`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:48.332546 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-0339de55-bb8a-428d-9e72-6ef184d16b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:48.523008 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac02f7a-e83b-4251-a67e-212ff9e674f8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:48.523325 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac02f7a-e83b-4251-a67e-212ff9e674f8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:48.523610 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac02f7a-e83b-4251-a67e-212ff9e674f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:48.540316 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 841/1666] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:48.545284 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339de55-bb8a-428d-9e72-6ef184d16b5b None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:48.623393 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-1ac02f7a-e83b-4251-a67e-212ff9e674f8 None None] MFA Rules not processed for user `3aa2e2a5a6ed4c5c9db91269266dd30b`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:48.625173 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1ac02f7a-e83b-4251-a67e-212ff9e674f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:48.774489 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 826/1667] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:48.797983 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac02f7a-e83b-4251-a67e-212ff9e674f8 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:48.799514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1ac02f7a-e83b-4251-a67e-212ff9e674f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:48.823292 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339de55-bb8a-428d-9e72-6ef184d16b5b None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:48.857225 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38149a18-4072-4297-8d8e-79364404cd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:48.859319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38149a18-4072-4297-8d8e-79364404cd0e admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:48.861197 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38149a18-4072-4297-8d8e-79364404cd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:48.861706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38149a18-4072-4297-8d8e-79364404cd0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:48.865056 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:48.867996 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38149a18-4072-4297-8d8e-79364404cd0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:48.877241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38149a18-4072-4297-8d8e-79364404cd0e admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:48.977731 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 842/1668] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:32:48 2022] POST /identity/v3/projects => generated 368 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:48.983732 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38149a18-4072-4297-8d8e-79364404cd0e admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:48.993565 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac02f7a-e83b-4251-a67e-212ff9e674f8 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:49.092171 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38149a18-4072-4297-8d8e-79364404cd0e admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:49.103442 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141d87c-3db0-44c0-a2db-1c752abbcc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:49.104646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141d87c-3db0-44c0-a2db-1c752abbcc97 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:49.105094 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141d87c-3db0-44c0-a2db-1c752abbcc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:49.105599 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141d87c-3db0-44c0-a2db-1c752abbcc97 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:49.107215 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141d87c-3db0-44c0-a2db-1c752abbcc97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:49.111177 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141d87c-3db0-44c0-a2db-1c752abbcc97 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:49.140895 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:49.175159 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 843/1669] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:32:48 2022] POST /identity/v3/users => generated 291 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:49.181477 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141d87c-3db0-44c0-a2db-1c752abbcc97 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:49.181679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877a922-a3b5-4e1b-92f3-7c97b058e389 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:49.182921 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877a922-a3b5-4e1b-92f3-7c97b058e389 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:49.183459 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877a922-a3b5-4e1b-92f3-7c97b058e389 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:49.184417 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877a922-a3b5-4e1b-92f3-7c97b058e389 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:49.217167 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141d87c-3db0-44c0-a2db-1c752abbcc97 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:49.239769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6b260-7cf5-4d09-be1a-b08c0681f267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:49.240547 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6b260-7cf5-4d09-be1a-b08c0681f267 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:49.241044 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6b260-7cf5-4d09-be1a-b08c0681f267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:49.241544 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6b260-7cf5-4d09-be1a-b08c0681f267 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:49.242904 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be6b260-7cf5-4d09-be1a-b08c0681f267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:49.251792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be6b260-7cf5-4d09-be1a-b08c0681f267 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:49.293987 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 844/1670] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:49 2022] GET /identity/v3/roles => generated 1427 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:32:49.295519 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6b260-7cf5-4d09-be1a-b08c0681f267 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:49.336425 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6b260-7cf5-4d09-be1a-b08c0681f267 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:49.357867 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c129e3-645f-4722-bb5a-f465bba1ab84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:49.359094 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c129e3-645f-4722-bb5a-f465bba1ab84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:49.359607 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c129e3-645f-4722-bb5a-f465bba1ab84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:49.360216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c129e3-645f-4722-bb5a-f465bba1ab84 admin admin] PATH_INFO: `/v3/projects/5e3c96d4f14d49a2b01f5996b65ca61b/users/4fb3a7a61346444ebb23ac4028dd7369/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:49.409522 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5877a922-a3b5-4e1b-92f3-7c97b058e389 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:49.412876 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5877a922-a3b5-4e1b-92f3-7c97b058e389 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:49.438320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c129e3-645f-4722-bb5a-f465bba1ab84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e3c96d4f14d49a2b01f5996b65ca61b, user_id=4fb3a7a61346444ebb23ac4028dd7369, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:49.444420 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c129e3-645f-4722-bb5a-f465bba1ab84 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:49.450452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 827/1671] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:32:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:32:49.493148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877a922-a3b5-4e1b-92f3-7c97b058e389 service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:49.498921 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 845/1672] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:32:49 2022] PUT /identity/v3/projects/5e3c96d4f14d49a2b01f5996b65ca61b/users/4fb3a7a61346444ebb23ac4028dd7369/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:49.534300 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877a922-a3b5-4e1b-92f3-7c97b058e389 service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:49.618853 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c129e3-645f-4722-bb5a-f465bba1ab84 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:49.626196 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-b8c129e3-645f-4722-bb5a-f465bba1ab84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:49.788637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:49.806468 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b8521-bbd6-4ca4-8eef-025e52ab6470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:49.807420 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b8521-bbd6-4ca4-8eef-025e52ab6470 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:49.807709 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b8521-bbd6-4ca4-8eef-025e52ab6470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:49.808023 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b8521-bbd6-4ca4-8eef-025e52ab6470 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:49.808749 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95b8521-bbd6-4ca4-8eef-025e52ab6470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:49.810596 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95b8521-bbd6-4ca4-8eef-025e52ab6470 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:49.832670 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 828/1673] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:49 2022] GET /identity/v3/roles => generated 1427 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:32:49.833575 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b8521-bbd6-4ca4-8eef-025e52ab6470 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:49.847447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d95b8521-bbd6-4ca4-8eef-025e52ab6470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:49.860728 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c129e3-645f-4722-bb5a-f465bba1ab84 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:49.877529 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b8521-bbd6-4ca4-8eef-025e52ab6470 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:49.877867 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a11cf7-757c-4392-a873-e1e756108c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:49.877867 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a11cf7-757c-4392-a873-e1e756108c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:49.877867 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a11cf7-757c-4392-a873-e1e756108c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:49.877867 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a11cf7-757c-4392-a873-e1e756108c94 admin admin] PATH_INFO: `/v3/projects/64e299419ec843c3be8a8e2d061f2ec2` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:49.879381 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a11cf7-757c-4392-a873-e1e756108c94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64e299419ec843c3be8a8e2d061f2ec2)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:49.880727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa5802-60b2-44cb-a266-1268663514ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:49.881802 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa5802-60b2-44cb-a266-1268663514ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:49.881802 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa5802-60b2-44cb-a266-1268663514ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:49.882220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa5802-60b2-44cb-a266-1268663514ac admin admin] PATH_INFO: `/v3/projects/7cec3109bb454e3ca7aa6362d721321d` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:49.883588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa5802-60b2-44cb-a266-1268663514ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cec3109bb454e3ca7aa6362d721321d)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:49.883823 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a11cf7-757c-4392-a873-e1e756108c94 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:49.885718 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa5802-60b2-44cb-a266-1268663514ac admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:49.966059 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8caa5802-60b2-44cb-a266-1268663514ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cec3109bb454e3ca7aa6362d721321d', 'request_id': 'req-8caa5802-60b2-44cb-a266-1268663514ac'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:49.976364 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8caa5802-60b2-44cb-a266-1268663514ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cec3109bb454e3ca7aa6362d721321d', 'request_id': 'req-8caa5802-60b2-44cb-a266-1268663514ac'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:49.985438 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8caa5802-60b2-44cb-a266-1268663514ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cec3109bb454e3ca7aa6362d721321d', 'request_id': 'req-8caa5802-60b2-44cb-a266-1268663514ac'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:50.037369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e9a11cf7-757c-4392-a873-e1e756108c94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64e299419ec843c3be8a8e2d061f2ec2', 'request_id': 'req-e9a11cf7-757c-4392-a873-e1e756108c94'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:50.037909 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8caa5802-60b2-44cb-a266-1268663514ac admin admin] The token cache is being invalidate because project 7cec3109bb454e3ca7aa6362d721321d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:50.038323 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8caa5802-60b2-44cb-a266-1268663514ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:50.038785 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8caa5802-60b2-44cb-a266-1268663514ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:50.040526 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 829/1674] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:32:49 2022] DELETE /identity/v3/projects/7cec3109bb454e3ca7aa6362d721321d => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:50.041376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa5802-60b2-44cb-a266-1268663514ac admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:50.045817 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8caa5802-60b2-44cb-a266-1268663514ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:50.056564 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e9a11cf7-757c-4392-a873-e1e756108c94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64e299419ec843c3be8a8e2d061f2ec2', 'request_id': 'req-e9a11cf7-757c-4392-a873-e1e756108c94'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:50.073349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e9a11cf7-757c-4392-a873-e1e756108c94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64e299419ec843c3be8a8e2d061f2ec2', 'request_id': 'req-e9a11cf7-757c-4392-a873-e1e756108c94'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:50.108078 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e9a11cf7-757c-4392-a873-e1e756108c94 admin admin] The token cache is being invalidate because project 64e299419ec843c3be8a8e2d061f2ec2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:50.108078 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e9a11cf7-757c-4392-a873-e1e756108c94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:50.108631 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e9a11cf7-757c-4392-a873-e1e756108c94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:50.113820 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 846/1675] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:32:49 2022] DELETE /identity/v3/projects/64e299419ec843c3be8a8e2d061f2ec2 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:32:50.113820 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f8c2d-c347-4306-a2c0-23b238d68ca8 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:50.113820 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f8c2d-c347-4306-a2c0-23b238d68ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:50.115809 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f8c2d-c347-4306-a2c0-23b238d68ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:50.169948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-3a4f8c2d-c347-4306-a2c0-23b238d68ca8 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:50.170688 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-3a4f8c2d-c347-4306-a2c0-23b238d68ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:50.221806 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa5802-60b2-44cb-a266-1268663514ac admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:50.288623 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:50.322294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 847/1676] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:50.323301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2aa8e-c0a9-4f5b-b0c3-c55a7f4de899 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:50.323590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2aa8e-c0a9-4f5b-b0c3-c55a7f4de899 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:50.323865 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2aa8e-c0a9-4f5b-b0c3-c55a7f4de899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:50.354315 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b794445-5a4e-40b3-b730-aa42749074b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:50.355789 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b794445-5a4e-40b3-b730-aa42749074b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:50.357421 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b794445-5a4e-40b3-b730-aa42749074b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:50.357723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b794445-5a4e-40b3-b730-aa42749074b0 admin admin] PATH_INFO: `/v3/projects/5e3c96d4f14d49a2b01f5996b65ca61b/users/4fb3a7a61346444ebb23ac4028dd7369/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:50.365192 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b794445-5a4e-40b3-b730-aa42749074b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e3c96d4f14d49a2b01f5996b65ca61b, user_id=4fb3a7a61346444ebb23ac4028dd7369, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:50.369771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-73f2aa8e-c0a9-4f5b-b0c3-c55a7f4de899 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:50.371053 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-73f2aa8e-c0a9-4f5b-b0c3-c55a7f4de899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:50.372378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b794445-5a4e-40b3-b730-aa42749074b0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:50.390051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 848/1677] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:50.391878 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7666eed-6c85-47db-8810-a167017f1ab1 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:50.395600 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7666eed-6c85-47db-8810-a167017f1ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:50.395902 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7666eed-6c85-47db-8810-a167017f1ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:50.396920 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 830/1678] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:32:50 2022] PUT /identity/v3/projects/5e3c96d4f14d49a2b01f5996b65ca61b/users/4fb3a7a61346444ebb23ac4028dd7369/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:50.399832 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faace26b-aeb8-45db-8896-bf17de3aacd5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:50.401797 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faace26b-aeb8-45db-8896-bf17de3aacd5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:50.402096 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faace26b-aeb8-45db-8896-bf17de3aacd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:50.440778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-d7666eed-6c85-47db-8810-a167017f1ab1 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:50.442136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d7666eed-6c85-47db-8810-a167017f1ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:50.445058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-faace26b-aeb8-45db-8896-bf17de3aacd5 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:50.447007 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-faace26b-aeb8-45db-8896-bf17de3aacd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:50.475058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:50.612008 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 831/1679] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:50.613377 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069054a-5579-4ada-aea7-fe2f7117cd9e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:50.615392 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069054a-5579-4ada-aea7-fe2f7117cd9e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:50.616344 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069054a-5579-4ada-aea7-fe2f7117cd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:50.651189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 849/1680] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:50.652259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7666eed-6c85-47db-8810-a167017f1ab1 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:50.681884 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7666eed-6c85-47db-8810-a167017f1ab1 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:50.692661 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34bedc1-4266-4f7a-8202-8b9a01b8594b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:50.693364 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bedc1-4266-4f7a-8202-8b9a01b8594b admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:50.693652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bedc1-4266-4f7a-8202-8b9a01b8594b admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:50.693760 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8069054a-5579-4ada-aea7-fe2f7117cd9e None None] MFA Rules not processed for user `4fb3a7a61346444ebb23ac4028dd7369`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:50.693931 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bedc1-4266-4f7a-8202-8b9a01b8594b admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:50.695115 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34bedc1-4266-4f7a-8202-8b9a01b8594b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:32:50.695115 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34bedc1-4266-4f7a-8202-8b9a01b8594b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:50.695299 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8069054a-5579-4ada-aea7-fe2f7117cd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:50.696675 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34bedc1-4266-4f7a-8202-8b9a01b8594b admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:50.724708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 850/1681] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:32:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:32:50.725754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34bedc1-4266-4f7a-8202-8b9a01b8594b admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:50.750230 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34bedc1-4266-4f7a-8202-8b9a01b8594b admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:50.770099 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11daecda-15ac-42d3-a3ce-27f62be47fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:50.770768 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11daecda-15ac-42d3-a3ce-27f62be47fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:50.771055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11daecda-15ac-42d3-a3ce-27f62be47fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:50.771341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11daecda-15ac-42d3-a3ce-27f62be47fa1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:50.772099 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11daecda-15ac-42d3-a3ce-27f62be47fa1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:32:50.772413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11daecda-15ac-42d3-a3ce-27f62be47fa1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:50.774106 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11daecda-15ac-42d3-a3ce-27f62be47fa1 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:50.804410 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 851/1682] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:32:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:32:50.805409 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11daecda-15ac-42d3-a3ce-27f62be47fa1 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:50.835794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11daecda-15ac-42d3-a3ce-27f62be47fa1 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:50.852887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1715b-b36c-487c-ad39-488412e6dca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:50.857126 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1715b-b36c-487c-ad39-488412e6dca3 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:50.858245 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1715b-b36c-487c-ad39-488412e6dca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:50.858245 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1715b-b36c-487c-ad39-488412e6dca3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:50.859738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1715b-b36c-487c-ad39-488412e6dca3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:50.870159 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1715b-b36c-487c-ad39-488412e6dca3 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:50.928063 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 832/1683] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:50 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:50.929009 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069054a-5579-4ada-aea7-fe2f7117cd9e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:50.954944 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 852/1684] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:32:50 2022] POST /identity/v3/projects => generated 374 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:50.957793 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1715b-b36c-487c-ad39-488412e6dca3 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:50.958089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069054a-5579-4ada-aea7-fe2f7117cd9e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:50.962405 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-fae1715b-b36c-487c-ad39-488412e6dca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:51.146679 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1715b-b36c-487c-ad39-488412e6dca3 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:51.161000 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333eb02d-3687-4581-a0a5-ea6472bbeaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:51.161795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333eb02d-3687-4581-a0a5-ea6472bbeaf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:51.161983 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333eb02d-3687-4581-a0a5-ea6472bbeaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:51.162239 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333eb02d-3687-4581-a0a5-ea6472bbeaf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:51.163771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333eb02d-3687-4581-a0a5-ea6472bbeaf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:51.163895 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8894d802-ae14-4d4a-abfc-e321b07b1fe5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:51.163895 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894d802-ae14-4d4a-abfc-e321b07b1fe5 service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:51.163895 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894d802-ae14-4d4a-abfc-e321b07b1fe5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:51.164273 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894d802-ae14-4d4a-abfc-e321b07b1fe5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:51.165201 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333eb02d-3687-4581-a0a5-ea6472bbeaf5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:51.223513 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 833/1685] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:32:50 2022] POST /identity/v3/projects => generated 364 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:51.224480 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333eb02d-3687-4581-a0a5-ea6472bbeaf5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:51.305599 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:51.347698 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:51.394064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8894d802-ae14-4d4a-abfc-e321b07b1fe5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:51.398286 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333eb02d-3687-4581-a0a5-ea6472bbeaf5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:51.399749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8894d802-ae14-4d4a-abfc-e321b07b1fe5 service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:51.414414 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa15292-ed11-4e9e-8299-a1e6abaf8d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:51.415420 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa15292-ed11-4e9e-8299-a1e6abaf8d6b admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:51.416458 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa15292-ed11-4e9e-8299-a1e6abaf8d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:51.416907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa15292-ed11-4e9e-8299-a1e6abaf8d6b admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:51.417962 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa15292-ed11-4e9e-8299-a1e6abaf8d6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:51.423462 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa15292-ed11-4e9e-8299-a1e6abaf8d6b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:51.425546 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 853/1686] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:32:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:51.426450 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8894d802-ae14-4d4a-abfc-e321b07b1fe5 service cinder] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:51.467337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8894d802-ae14-4d4a-abfc-e321b07b1fe5 service cinder] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:51.488551 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 834/1687] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:32:51 2022] POST /identity/v3/users => generated 288 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:51.492246 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa15292-ed11-4e9e-8299-a1e6abaf8d6b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:51.495077 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e1a49-4b7e-4ce9-9e33-de9950fad702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:51.495806 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e1a49-4b7e-4ce9-9e33-de9950fad702 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:51.496333 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e1a49-4b7e-4ce9-9e33-de9950fad702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:51.496516 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e1a49-4b7e-4ce9-9e33-de9950fad702 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:51.497178 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e1a49-4b7e-4ce9-9e33-de9950fad702 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:51.499458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e1a49-4b7e-4ce9-9e33-de9950fad702 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:51.517263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa15292-ed11-4e9e-8299-a1e6abaf8d6b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:51.531780 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028cdc9f-4766-44d8-bcd9-20054cbf9c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:51.531780 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028cdc9f-4766-44d8-bcd9-20054cbf9c06 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:51.532299 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028cdc9f-4766-44d8-bcd9-20054cbf9c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:51.532299 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028cdc9f-4766-44d8-bcd9-20054cbf9c06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:51.533859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028cdc9f-4766-44d8-bcd9-20054cbf9c06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:51.537630 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028cdc9f-4766-44d8-bcd9-20054cbf9c06 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:51.552213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 854/1688] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:32:51 2022] POST /identity/v3/users => generated 283 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:51.558492 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e1a49-4b7e-4ce9-9e33-de9950fad702 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:51.584392 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 835/1689] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:51 2022] GET /identity/v3/roles => generated 1427 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:51.585430 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028cdc9f-4766-44d8-bcd9-20054cbf9c06 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:51.586711 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e1a49-4b7e-4ce9-9e33-de9950fad702 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:51.593205 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc019ec7-6bfc-4ebb-9924-4f2685eb50a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:51.594103 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc019ec7-6bfc-4ebb-9924-4f2685eb50a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:51.594632 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc019ec7-6bfc-4ebb-9924-4f2685eb50a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:51.594921 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc019ec7-6bfc-4ebb-9924-4f2685eb50a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:51.595706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc019ec7-6bfc-4ebb-9924-4f2685eb50a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:51.597763 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc019ec7-6bfc-4ebb-9924-4f2685eb50a2 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:51.608091 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028cdc9f-4766-44d8-bcd9-20054cbf9c06 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:51.613930 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 855/1690] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:51 2022] GET /identity/v3/roles => generated 1427 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:51.619605 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc019ec7-6bfc-4ebb-9924-4f2685eb50a2 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:51.621877 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765384e9-c609-422c-ab86-9650c1da3ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:51.622601 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765384e9-c609-422c-ab86-9650c1da3ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:51.622883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765384e9-c609-422c-ab86-9650c1da3ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:51.623154 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765384e9-c609-422c-ab86-9650c1da3ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:51.623865 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765384e9-c609-422c-ab86-9650c1da3ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:51.625545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765384e9-c609-422c-ab86-9650c1da3ec3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:51.632131 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc019ec7-6bfc-4ebb-9924-4f2685eb50a2 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:51.642908 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5f0ed-a97c-479d-9edb-79f067c8ed03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:51.645517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5f0ed-a97c-479d-9edb-79f067c8ed03 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:51.647687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5f0ed-a97c-479d-9edb-79f067c8ed03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:51.648126 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5f0ed-a97c-479d-9edb-79f067c8ed03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:51.651547 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed5f0ed-a97c-479d-9edb-79f067c8ed03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:51.654017 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 836/1691] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:51 2022] GET /identity/v3/roles => generated 1427 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:51.655119 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765384e9-c609-422c-ab86-9650c1da3ec3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:51.664800 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed5f0ed-a97c-479d-9edb-79f067c8ed03 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:51.685293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765384e9-c609-422c-ab86-9650c1da3ec3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:51.693534 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 856/1692] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:51 2022] GET /identity/v3/roles => generated 1427 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:51.700246 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5f0ed-a97c-479d-9edb-79f067c8ed03 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:51.702650 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c59843-091b-49e8-b4bb-bd859f5b71c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:51.703425 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c59843-091b-49e8-b4bb-bd859f5b71c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:51.703712 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c59843-091b-49e8-b4bb-bd859f5b71c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:51.704093 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c59843-091b-49e8-b4bb-bd859f5b71c3 admin admin] PATH_INFO: `/v3/projects/cd8d7f51f7ab4e0f8a86effcc1643dfe/users/e0984ca333794b0d9b04e61fe13c2485/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:51.715220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5f0ed-a97c-479d-9edb-79f067c8ed03 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:51.721135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b07a0-998c-493c-bac9-01b9ed0524a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:51.722099 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b07a0-998c-493c-bac9-01b9ed0524a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:51.722496 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b07a0-998c-493c-bac9-01b9ed0524a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:51.723835 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b07a0-998c-493c-bac9-01b9ed0524a6 admin admin] PATH_INFO: `/v3/projects/53b59a20da964076b08dfa6facd27e6e/users/7f2ef459165f4ddaa7233346db66eccd/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:51.749345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c59843-091b-49e8-b4bb-bd859f5b71c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd8d7f51f7ab4e0f8a86effcc1643dfe, user_id=e0984ca333794b0d9b04e61fe13c2485, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:51.754405 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c59843-091b-49e8-b4bb-bd859f5b71c3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:51.768138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113b07a0-998c-493c-bac9-01b9ed0524a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53b59a20da964076b08dfa6facd27e6e, user_id=7f2ef459165f4ddaa7233346db66eccd, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:51.776236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113b07a0-998c-493c-bac9-01b9ed0524a6 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:51.789955 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 837/1693] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:32:51 2022] PUT /identity/v3/projects/cd8d7f51f7ab4e0f8a86effcc1643dfe/users/e0984ca333794b0d9b04e61fe13c2485/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:51.791658 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb29431-5ced-4f08-b519-fc4c4c8eeada None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:51.793825 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb29431-5ced-4f08-b519-fc4c4c8eeada None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:51.794157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb29431-5ced-4f08-b519-fc4c4c8eeada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:51.840402 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7eb29431-5ced-4f08-b519-fc4c4c8eeada None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:51.844961 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7eb29431-5ced-4f08-b519-fc4c4c8eeada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:51.854713 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 857/1694] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:32:51 2022] PUT /identity/v3/projects/53b59a20da964076b08dfa6facd27e6e/users/7f2ef459165f4ddaa7233346db66eccd/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:51.878924 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147ded3-edb7-4d5d-bcb3-bef20fc3a8ec None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:51.881483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147ded3-edb7-4d5d-bcb3-bef20fc3a8ec None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:51.883583 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147ded3-edb7-4d5d-bcb3-bef20fc3a8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:52.010022 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 838/1695] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:52.022625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-e147ded3-edb7-4d5d-bcb3-bef20fc3a8ec None None] MFA Rules not processed for user `7f2ef459165f4ddaa7233346db66eccd`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:52.026989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb29431-5ced-4f08-b519-fc4c4c8eeada None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:52.030156 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7eb29431-5ced-4f08-b519-fc4c4c8eeada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:52.031360 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-e147ded3-edb7-4d5d-bcb3-bef20fc3a8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:52.172625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb29431-5ced-4f08-b519-fc4c4c8eeada None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:52.184497 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0d3e5-2df8-41bc-82ad-4684dd7c8bbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:52.185259 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0d3e5-2df8-41bc-82ad-4684dd7c8bbe service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:52.185495 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0d3e5-2df8-41bc-82ad-4684dd7c8bbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:52.185750 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0d3e5-2df8-41bc-82ad-4684dd7c8bbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:52.232125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0d3e5-2df8-41bc-82ad-4684dd7c8bbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:52.234245 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0d3e5-2df8-41bc-82ad-4684dd7c8bbe service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:52.256573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 858/1696] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:52.260891 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147ded3-edb7-4d5d-bcb3-bef20fc3a8ec None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:52.279609 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147ded3-edb7-4d5d-bcb3-bef20fc3a8ec None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:52.287217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 839/1697] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:32:52 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:52.288620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa97a4-faee-438d-a86c-13adbd733842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:52.289660 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa97a4-faee-438d-a86c-13adbd733842 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:52.289959 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa97a4-faee-438d-a86c-13adbd733842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:52.290663 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa97a4-faee-438d-a86c-13adbd733842 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:52.291726 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa97a4-faee-438d-a86c-13adbd733842 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:52.294790 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa97a4-faee-438d-a86c-13adbd733842 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:52.346821 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 859/1698] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:32:52 2022] POST /identity/v3/projects => generated 364 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:32:52.351133 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0d3e5-2df8-41bc-82ad-4684dd7c8bbe service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:52.368661 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0d3e5-2df8-41bc-82ad-4684dd7c8bbe service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:52.404214 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:52.502804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a35d08-b6a9-4643-815d-c8cee9ffec80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:52.503647 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a35d08-b6a9-4643-815d-c8cee9ffec80 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:52.504013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a35d08-b6a9-4643-815d-c8cee9ffec80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:52.504327 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a35d08-b6a9-4643-815d-c8cee9ffec80 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:52.504982 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a35d08-b6a9-4643-815d-c8cee9ffec80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:52.524339 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a35d08-b6a9-4643-815d-c8cee9ffec80 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:52.607753 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 840/1699] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:32:52 2022] POST /identity/v3/users => generated 289 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:52.611537 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa97a4-faee-438d-a86c-13adbd733842 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:52.642938 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa97a4-faee-438d-a86c-13adbd733842 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:52.657025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9ac24-99d3-4a47-9778-4a195530d059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:52.657025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9ac24-99d3-4a47-9778-4a195530d059 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:52.657279 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9ac24-99d3-4a47-9778-4a195530d059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:52.663252 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9ac24-99d3-4a47-9778-4a195530d059 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:52.665238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9ac24-99d3-4a47-9778-4a195530d059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:52.667033 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9ac24-99d3-4a47-9778-4a195530d059 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:52.713936 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a35d08-b6a9-4643-815d-c8cee9ffec80 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:52.735430 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 860/1700] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:52 2022] GET /identity/v3/roles => generated 1427 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:52.741534 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a35d08-b6a9-4643-815d-c8cee9ffec80 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:52.753840 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f0889-87f2-4e5a-b028-c133efc41029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:52.755812 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f0889-87f2-4e5a-b028-c133efc41029 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:52.757411 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f0889-87f2-4e5a-b028-c133efc41029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:52.757809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f0889-87f2-4e5a-b028-c133efc41029 admin admin] PATH_INFO: `/v3/projects/2c7d8b766dd644cbb1aa6b0af0ff5895/users/9d5d847d4afd4a9fac48b4a54cf37e40/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:52.817654 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f0889-87f2-4e5a-b028-c133efc41029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c7d8b766dd644cbb1aa6b0af0ff5895, user_id=9d5d847d4afd4a9fac48b4a54cf37e40, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:52.821380 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f0889-87f2-4e5a-b028-c133efc41029 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:52.849105 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 841/1701] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:32:52 2022] PUT /identity/v3/projects/2c7d8b766dd644cbb1aa6b0af0ff5895/users/9d5d847d4afd4a9fac48b4a54cf37e40/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:32:52.853780 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9ac24-99d3-4a47-9778-4a195530d059 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:52.870250 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9ac24-99d3-4a47-9778-4a195530d059 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:52.969038 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:53.006175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1227d-1b9a-4c46-967a-4b9fd51d9fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:53.006991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1227d-1b9a-4c46-967a-4b9fd51d9fe0 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:53.007384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1227d-1b9a-4c46-967a-4b9fd51d9fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:53.007771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1227d-1b9a-4c46-967a-4b9fd51d9fe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:53.008659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f1227d-1b9a-4c46-967a-4b9fd51d9fe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:53.011820 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f1227d-1b9a-4c46-967a-4b9fd51d9fe0 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:53.043708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 861/1702] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:52 2022] GET /identity/v3/roles => generated 1427 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:53.045239 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f0889-87f2-4e5a-b028-c133efc41029 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:53.055968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f0889-87f2-4e5a-b028-c133efc41029 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:53.063415 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cbcffb-1604-45fc-9b38-8f90e58cc219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:53.064277 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbcffb-1604-45fc-9b38-8f90e58cc219 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:53.064493 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbcffb-1604-45fc-9b38-8f90e58cc219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:53.064774 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbcffb-1604-45fc-9b38-8f90e58cc219 admin admin] PATH_INFO: `/v3/projects/2c7d8b766dd644cbb1aa6b0af0ff5895/users/9d5d847d4afd4a9fac48b4a54cf37e40/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:53.069739 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cbcffb-1604-45fc-9b38-8f90e58cc219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c7d8b766dd644cbb1aa6b0af0ff5895, user_id=9d5d847d4afd4a9fac48b4a54cf37e40, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:53.075312 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cbcffb-1604-45fc-9b38-8f90e58cc219 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:53.107398 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 842/1703] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:32:53 2022] PUT /identity/v3/projects/2c7d8b766dd644cbb1aa6b0af0ff5895/users/9d5d847d4afd4a9fac48b4a54cf37e40/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:53.120507 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff4ee3-d8db-4802-875a-f41e801b11ab None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:53.120883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff4ee3-d8db-4802-875a-f41e801b11ab None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:53.122190 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff4ee3-d8db-4802-875a-f41e801b11ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:53.207885 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-e9ff4ee3-d8db-4802-875a-f41e801b11ab None None] MFA Rules not processed for user `9d5d847d4afd4a9fac48b4a54cf37e40`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:53.212140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-e9ff4ee3-d8db-4802-875a-f41e801b11ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:53.442807 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 862/1704] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:53.462772 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cbcffb-1604-45fc-9b38-8f90e58cc219 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:53.464632 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-28cbcffb-1604-45fc-9b38-8f90e58cc219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:53.645245 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cbcffb-1604-45fc-9b38-8f90e58cc219 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:53.659545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f03f4b-47a9-47ec-b309-2ea036698c2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:53.660491 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f03f4b-47a9-47ec-b309-2ea036698c2e service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:53.660849 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f03f4b-47a9-47ec-b309-2ea036698c2e service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:53.661404 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f03f4b-47a9-47ec-b309-2ea036698c2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:53.680850 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f03f4b-47a9-47ec-b309-2ea036698c2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:53.682529 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f03f4b-47a9-47ec-b309-2ea036698c2e service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:53.723093 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 843/1705] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:32:53 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:54.080312 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff4ee3-d8db-4802-875a-f41e801b11ab None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:54.092101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff4ee3-d8db-4802-875a-f41e801b11ab None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:54.105271 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468b9bc-f6eb-4ba0-af80-a4a1469b9068 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:54.106265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468b9bc-f6eb-4ba0-af80-a4a1469b9068 service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:54.106698 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468b9bc-f6eb-4ba0-af80-a4a1469b9068 service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:54.107049 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468b9bc-f6eb-4ba0-af80-a4a1469b9068 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:54.283823 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468b9bc-f6eb-4ba0-af80-a4a1469b9068 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:54.286643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468b9bc-f6eb-4ba0-af80-a4a1469b9068 service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:54.321133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 863/1706] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:32:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:55.292275 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f03f4b-47a9-47ec-b309-2ea036698c2e service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:55.295120 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c5f03f4b-47a9-47ec-b309-2ea036698c2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:55.461399 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f03f4b-47a9-47ec-b309-2ea036698c2e service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:55.470850 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69838c07-e1dd-4cc8-bc0e-98822517af09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:55.471830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69838c07-e1dd-4cc8-bc0e-98822517af09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:55.472298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69838c07-e1dd-4cc8-bc0e-98822517af09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:55.472758 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69838c07-e1dd-4cc8-bc0e-98822517af09 admin admin] PATH_INFO: `/v3/users/98ab41698d2841aeafafc2b4d0ab8d26` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:55.476906 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69838c07-e1dd-4cc8-bc0e-98822517af09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98ab41698d2841aeafafc2b4d0ab8d26)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:55.480856 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69838c07-e1dd-4cc8-bc0e-98822517af09 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:55.526957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:55.646788 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-69838c07-e1dd-4cc8-bc0e-98822517af09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98ab41698d2841aeafafc2b4d0ab8d26'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:55.682217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-69838c07-e1dd-4cc8-bc0e-98822517af09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98ab41698d2841aeafafc2b4d0ab8d26'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:55.683379 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-69838c07-e1dd-4cc8-bc0e-98822517af09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98ab41698d2841aeafafc2b4d0ab8d26'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:55.710191 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-69838c07-e1dd-4cc8-bc0e-98822517af09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98ab41698d2841aeafafc2b4d0ab8d26'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:55.717565 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-69838c07-e1dd-4cc8-bc0e-98822517af09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98ab41698d2841aeafafc2b4d0ab8d26'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:55.747914 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 844/1707] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:32:55 2022] DELETE /identity/v3/users/98ab41698d2841aeafafc2b4d0ab8d26 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:55.756174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bc9d3-fe90-4b94-a434-3b5d6b4eec92 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:55.756174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bc9d3-fe90-4b94-a434-3b5d6b4eec92 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:55.756174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bc9d3-fe90-4b94-a434-3b5d6b4eec92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:55.823332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-303bc9d3-fe90-4b94-a434-3b5d6b4eec92 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:55.827108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-303bc9d3-fe90-4b94-a434-3b5d6b4eec92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:56.033307 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:56.055828 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 864/1708] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:55 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:56.131306 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303bc9d3-fe90-4b94-a434-3b5d6b4eec92 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:56.134057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6406a-8b67-413b-897f-3f4b5a19c5d4 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:56.134057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6406a-8b67-413b-897f-3f4b5a19c5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:56.134254 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6406a-8b67-413b-897f-3f4b5a19c5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:56.135196 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-303bc9d3-fe90-4b94-a434-3b5d6b4eec92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:56.247584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a3b6406a-8b67-413b-897f-3f4b5a19c5d4 None None] MFA Rules not processed for user `e0984ca333794b0d9b04e61fe13c2485`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:56.249762 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a3b6406a-8b67-413b-897f-3f4b5a19c5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:56.297337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303bc9d3-fe90-4b94-a434-3b5d6b4eec92 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:56.304325 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbd583-f072-400f-ae54-d9cfda356e94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:56.305177 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbd583-f072-400f-ae54-d9cfda356e94 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:56.305608 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbd583-f072-400f-ae54-d9cfda356e94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:56.306060 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbd583-f072-400f-ae54-d9cfda356e94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:56.321454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cbd583-f072-400f-ae54-d9cfda356e94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:56.325441 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cbd583-f072-400f-ae54-d9cfda356e94 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:56.351574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 865/1709] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:32:56 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:56.429077 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e608d-88b4-4850-8221-d089853123bc None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:56.430616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e608d-88b4-4850-8221-d089853123bc None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:56.430616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e608d-88b4-4850-8221-d089853123bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:56.523144 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-5c1e608d-88b4-4850-8221-d089853123bc None None] MFA Rules not processed for user `41a38b5af1654f44bc6cb5832c2e8e87`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:56.525363 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 845/1710] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:56.526493 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-5c1e608d-88b4-4850-8221-d089853123bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:56.531183 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6406a-8b67-413b-897f-3f4b5a19c5d4 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:56.538816 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a3b6406a-8b67-413b-897f-3f4b5a19c5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:56.613435 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6406a-8b67-413b-897f-3f4b5a19c5d4 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:56.624347 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dca47f-ff97-41be-941a-c17c29277d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:56.627148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dca47f-ff97-41be-941a-c17c29277d14 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:56.627467 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dca47f-ff97-41be-941a-c17c29277d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:56.627677 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dca47f-ff97-41be-941a-c17c29277d14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:56.628897 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dca47f-ff97-41be-941a-c17c29277d14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:56.636424 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dca47f-ff97-41be-941a-c17c29277d14 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:56.714819 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 846/1711] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:32:56 2022] POST /identity/v3/projects => generated 372 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:56.718461 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dca47f-ff97-41be-941a-c17c29277d14 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:56.736641 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 866/1712] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:32:56.752764 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e608d-88b4-4850-8221-d089853123bc None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:56.763022 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-5c1e608d-88b4-4850-8221-d089853123bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:57.042253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:57.064562 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dca47f-ff97-41be-941a-c17c29277d14 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:57.092283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c2f1a-6ad1-4721-8ba0-af18b315eca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:57.092967 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c2f1a-6ad1-4721-8ba0-af18b315eca0 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:57.100715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c2f1a-6ad1-4721-8ba0-af18b315eca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:57.101220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c2f1a-6ad1-4721-8ba0-af18b315eca0 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:57.101997 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c2f1a-6ad1-4721-8ba0-af18b315eca0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:57.118131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c2f1a-6ad1-4721-8ba0-af18b315eca0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:57.198377 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 847/1713] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:32:56 2022] POST /identity/v3/users => generated 293 bytes in 479 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:32:57.198377 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c2f1a-6ad1-4721-8ba0-af18b315eca0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:57.223898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e608d-88b4-4850-8221-d089853123bc None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:57.234483 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c2f1a-6ad1-4721-8ba0-af18b315eca0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:57.265947 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47deb17-4daf-4bd3-aaac-8d6bb6f8135d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:57.266687 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47deb17-4daf-4bd3-aaac-8d6bb6f8135d admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:57.266980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47deb17-4daf-4bd3-aaac-8d6bb6f8135d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:57.267268 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47deb17-4daf-4bd3-aaac-8d6bb6f8135d admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:57.268039 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47deb17-4daf-4bd3-aaac-8d6bb6f8135d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:57.274172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47deb17-4daf-4bd3-aaac-8d6bb6f8135d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:57.283003 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:57.305277 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 848/1714] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:57 2022] GET /identity/v3/roles => generated 1427 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:57.309818 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47deb17-4daf-4bd3-aaac-8d6bb6f8135d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:57.317678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b47deb17-4daf-4bd3-aaac-8d6bb6f8135d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:57.356416 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47deb17-4daf-4bd3-aaac-8d6bb6f8135d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:57.356658 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fde878-8a3b-4373-86ac-2ba18c7b7e40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:57.357141 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fde878-8a3b-4373-86ac-2ba18c7b7e40 service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:57.357386 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fde878-8a3b-4373-86ac-2ba18c7b7e40 service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:57.357691 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fde878-8a3b-4373-86ac-2ba18c7b7e40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:57.379237 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bbc597-f943-4ce1-9d7e-116aa4791435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:57.380378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bbc597-f943-4ce1-9d7e-116aa4791435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:57.380914 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bbc597-f943-4ce1-9d7e-116aa4791435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:57.381353 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bbc597-f943-4ce1-9d7e-116aa4791435 admin admin] PATH_INFO: `/v3/users/2ac28f9efd59480c8c6f5385e5945b25` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:57.383723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bbc597-f943-4ce1-9d7e-116aa4791435 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ac28f9efd59480c8c6f5385e5945b25)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:57.386125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bbc597-f943-4ce1-9d7e-116aa4791435 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:57.462857 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fde878-8a3b-4373-86ac-2ba18c7b7e40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:57.465092 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fde878-8a3b-4373-86ac-2ba18c7b7e40 service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:57.494715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f1bbc597-f943-4ce1-9d7e-116aa4791435 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ac28f9efd59480c8c6f5385e5945b25'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:57.503112 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f1bbc597-f943-4ce1-9d7e-116aa4791435 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ac28f9efd59480c8c6f5385e5945b25'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:57.503899 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f1bbc597-f943-4ce1-9d7e-116aa4791435 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ac28f9efd59480c8c6f5385e5945b25'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:57.515999 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 867/1715] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:32:56 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 763 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:32:57.520956 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fde878-8a3b-4373-86ac-2ba18c7b7e40 service nova] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:57.551355 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f1bbc597-f943-4ce1-9d7e-116aa4791435 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ac28f9efd59480c8c6f5385e5945b25'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:57.552253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f1bbc597-f943-4ce1-9d7e-116aa4791435 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ac28f9efd59480c8c6f5385e5945b25'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:57.577263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fde878-8a3b-4373-86ac-2ba18c7b7e40 service nova] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:57.591645 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 849/1716] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:32:57 2022] DELETE /identity/v3/users/2ac28f9efd59480c8c6f5385e5945b25 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:57.592794 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bbc597-f943-4ce1-9d7e-116aa4791435 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:57.595969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f1bbc597-f943-4ce1-9d7e-116aa4791435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:57.743692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12620a-eed1-4505-ae6d-dd02d155e792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:57.745133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12620a-eed1-4505-ae6d-dd02d155e792 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:57.746559 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12620a-eed1-4505-ae6d-dd02d155e792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:57.746868 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12620a-eed1-4505-ae6d-dd02d155e792 admin admin] PATH_INFO: `/v3/projects/c99bb9fb09664e279cf4450ad416b576/users/a0d4f49c68d3442f89c33b3f7f9fd9aa/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:57.776787 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bbc597-f943-4ce1-9d7e-116aa4791435 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:57.784269 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12620a-eed1-4505-ae6d-dd02d155e792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c99bb9fb09664e279cf4450ad416b576, user_id=a0d4f49c68d3442f89c33b3f7f9fd9aa, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:57.791650 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12620a-eed1-4505-ae6d-dd02d155e792 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:57.799102 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9505e2f8-c0c9-4d67-b225-27ed926235cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:57.799804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505e2f8-c0c9-4d67-b225-27ed926235cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:57.800124 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505e2f8-c0c9-4d67-b225-27ed926235cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:57.800410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505e2f8-c0c9-4d67-b225-27ed926235cf admin admin] PATH_INFO: `/v3/projects/dc5cb824051a46cea9fa28748570cf8d` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:57.805713 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9505e2f8-c0c9-4d67-b225-27ed926235cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc5cb824051a46cea9fa28748570cf8d)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:57.807815 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9505e2f8-c0c9-4d67-b225-27ed926235cf admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:57.834630 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 868/1717] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:32:57 2022] PUT /identity/v3/projects/c99bb9fb09664e279cf4450ad416b576/users/a0d4f49c68d3442f89c33b3f7f9fd9aa/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:57.838336 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d593c8-f0c3-4fdd-b5f8-c11958b4491b None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:57.839623 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d593c8-f0c3-4fdd-b5f8-c11958b4491b None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:57.839755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d593c8-f0c3-4fdd-b5f8-c11958b4491b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:57.871267 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-65d593c8-f0c3-4fdd-b5f8-c11958b4491b None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:57.875376 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-65d593c8-f0c3-4fdd-b5f8-c11958b4491b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:57.989291 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9505e2f8-c0c9-4d67-b225-27ed926235cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc5cb824051a46cea9fa28748570cf8d', 'request_id': 'req-9505e2f8-c0c9-4d67-b225-27ed926235cf'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:57.990862 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=82,keystone:INSERT=4,keystone:DELETE=14 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:58.013345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9505e2f8-c0c9-4d67-b225-27ed926235cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc5cb824051a46cea9fa28748570cf8d', 'request_id': 'req-9505e2f8-c0c9-4d67-b225-27ed926235cf'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:58.024430 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:58.028755 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9505e2f8-c0c9-4d67-b225-27ed926235cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc5cb824051a46cea9fa28748570cf8d', 'request_id': 'req-9505e2f8-c0c9-4d67-b225-27ed926235cf'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:58.051750 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9505e2f8-c0c9-4d67-b225-27ed926235cf admin admin] The token cache is being invalidate because project dc5cb824051a46cea9fa28748570cf8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:32:58.052217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9505e2f8-c0c9-4d67-b225-27ed926235cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:58.054417 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9505e2f8-c0c9-4d67-b225-27ed926235cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:58.056752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 850/1718] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:32:57 2022] DELETE /identity/v3/projects/dc5cb824051a46cea9fa28748570cf8d => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:58.057445 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9505e2f8-c0c9-4d67-b225-27ed926235cf admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:58.064166 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 869/1719] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:57 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:58.067796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9505e2f8-c0c9-4d67-b225-27ed926235cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:58.067893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d593c8-f0c3-4fdd-b5f8-c11958b4491b None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:58.090617 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d593c8-f0c3-4fdd-b5f8-c11958b4491b None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:58.327242 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9505e2f8-c0c9-4d67-b225-27ed926235cf admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:58.332200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae5f0b-9c55-4dc1-a63f-ed79b55b0397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:58.332899 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae5f0b-9c55-4dc1-a63f-ed79b55b0397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:58.333184 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae5f0b-9c55-4dc1-a63f-ed79b55b0397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:58.333468 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae5f0b-9c55-4dc1-a63f-ed79b55b0397 admin admin] PATH_INFO: `/v3/users/4fb3a7a61346444ebb23ac4028dd7369` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:58.337776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae5f0b-9c55-4dc1-a63f-ed79b55b0397 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fb3a7a61346444ebb23ac4028dd7369)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:58.351437 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae5f0b-9c55-4dc1-a63f-ed79b55b0397 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:58.358375 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c69c4b-8ecc-4c26-bbf2-59ca31bdfd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:58.359078 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c69c4b-8ecc-4c26-bbf2-59ca31bdfd51 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:58.359373 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c69c4b-8ecc-4c26-bbf2-59ca31bdfd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:58.359656 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c69c4b-8ecc-4c26-bbf2-59ca31bdfd51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:58.360591 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c69c4b-8ecc-4c26-bbf2-59ca31bdfd51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:58.369443 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c69c4b-8ecc-4c26-bbf2-59ca31bdfd51 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:58.400118 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 851/1720] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:32:58 2022] GET /identity/v3/roles => generated 1427 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:32:58.403531 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c69c4b-8ecc-4c26-bbf2-59ca31bdfd51 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:58.411257 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b0c69c4b-8ecc-4c26-bbf2-59ca31bdfd51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:58.488745 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-72ae5f0b-9c55-4dc1-a63f-ed79b55b0397 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fb3a7a61346444ebb23ac4028dd7369'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:58.492639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-72ae5f0b-9c55-4dc1-a63f-ed79b55b0397 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fb3a7a61346444ebb23ac4028dd7369'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:58.536059 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-72ae5f0b-9c55-4dc1-a63f-ed79b55b0397 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fb3a7a61346444ebb23ac4028dd7369'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:58.572667 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-72ae5f0b-9c55-4dc1-a63f-ed79b55b0397 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fb3a7a61346444ebb23ac4028dd7369'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:58.605433 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-72ae5f0b-9c55-4dc1-a63f-ed79b55b0397 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fb3a7a61346444ebb23ac4028dd7369'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:58.613492 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 870/1721] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:32:58 2022] DELETE /identity/v3/users/4fb3a7a61346444ebb23ac4028dd7369 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:58.617465 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae5f0b-9c55-4dc1-a63f-ed79b55b0397 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:58.624963 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-72ae5f0b-9c55-4dc1-a63f-ed79b55b0397 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:58.635913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c69c4b-8ecc-4c26-bbf2-59ca31bdfd51 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:58.758231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e67c1d-b381-4783-9d08-3f862d3cb3b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:58.760435 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e67c1d-b381-4783-9d08-3f862d3cb3b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:58.760895 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e67c1d-b381-4783-9d08-3f862d3cb3b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:58.761336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e67c1d-b381-4783-9d08-3f862d3cb3b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:58.763733 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-78e67c1d-b381-4783-9d08-3f862d3cb3b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:58.861804 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae5f0b-9c55-4dc1-a63f-ed79b55b0397 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:58.873669 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:58.885303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451ed59-d095-4781-bb44-ceff840d0e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:58.886486 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451ed59-d095-4781-bb44-ceff840d0e17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:58.886976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451ed59-d095-4781-bb44-ceff840d0e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:58.887890 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451ed59-d095-4781-bb44-ceff840d0e17 admin admin] PATH_INFO: `/v3/projects/c99bb9fb09664e279cf4450ad416b576/users/a0d4f49c68d3442f89c33b3f7f9fd9aa/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:58.896344 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451ed59-d095-4781-bb44-ceff840d0e17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c99bb9fb09664e279cf4450ad416b576, user_id=a0d4f49c68d3442f89c33b3f7f9fd9aa, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:58.900120 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451ed59-d095-4781-bb44-ceff840d0e17 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:58.908449 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e67c1d-b381-4783-9d08-3f862d3cb3b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:58.910215 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e67c1d-b381-4783-9d08-3f862d3cb3b8 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:58.928700 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 852/1722] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:32:58 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:32:58.930922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e67c1d-b381-4783-9d08-3f862d3cb3b8 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:58.933695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-78e67c1d-b381-4783-9d08-3f862d3cb3b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:58.934040 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 871/1723] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:32:58 2022] PUT /identity/v3/projects/c99bb9fb09664e279cf4450ad416b576/users/a0d4f49c68d3442f89c33b3f7f9fd9aa/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:58.938736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dca592-c07f-4b31-b46a-fea4ee123f0a None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:58.939030 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dca592-c07f-4b31-b46a-fea4ee123f0a None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:58.941142 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dca592-c07f-4b31-b46a-fea4ee123f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:58.958426 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:32:58.975077 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e67c1d-b381-4783-9d08-3f862d3cb3b8 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:59.016715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-27dca592-c07f-4b31-b46a-fea4ee123f0a None None] MFA Rules not processed for user `3aa2e2a5a6ed4c5c9db91269266dd30b`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:59.018115 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-27dca592-c07f-4b31-b46a-fea4ee123f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:59.095720 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70c600-8373-44ba-8c39-26e10872257d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:59.096254 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70c600-8373-44ba-8c39-26e10872257d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:59.096516 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70c600-8373-44ba-8c39-26e10872257d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:59.096795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70c600-8373-44ba-8c39-26e10872257d admin admin] PATH_INFO: `/v3/users/7f2ef459165f4ddaa7233346db66eccd` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:59.098946 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e70c600-8373-44ba-8c39-26e10872257d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f2ef459165f4ddaa7233346db66eccd)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:59.100895 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e70c600-8373-44ba-8c39-26e10872257d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:59.203611 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2e70c600-8373-44ba-8c39-26e10872257d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f2ef459165f4ddaa7233346db66eccd'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:59.212000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 872/1724] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:59.252873 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2e70c600-8373-44ba-8c39-26e10872257d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f2ef459165f4ddaa7233346db66eccd'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:59.255424 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2e70c600-8373-44ba-8c39-26e10872257d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f2ef459165f4ddaa7233346db66eccd'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:59.312361 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2e70c600-8373-44ba-8c39-26e10872257d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f2ef459165f4ddaa7233346db66eccd'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:59.313824 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2e70c600-8373-44ba-8c39-26e10872257d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f2ef459165f4ddaa7233346db66eccd'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:32:59.356373 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 853/1725] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:32:58 2022] DELETE /identity/v3/users/7f2ef459165f4ddaa7233346db66eccd => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:32:59.358148 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772723a-cf06-436b-aafd-6ff95fe7b21d None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:59.358445 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772723a-cf06-436b-aafd-6ff95fe7b21d None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:59.358728 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772723a-cf06-436b-aafd-6ff95fe7b21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:59.412008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-d772723a-cf06-436b-aafd-6ff95fe7b21d None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:32:59.413867 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d772723a-cf06-436b-aafd-6ff95fe7b21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:59.572610 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 873/1726] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:32:59 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:32:59.634325 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70c600-8373-44ba-8c39-26e10872257d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:32:59.636163 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2e70c600-8373-44ba-8c39-26e10872257d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:32:59.754642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70c600-8373-44ba-8c39-26e10872257d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:32:59.761916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7ca6f-7800-4e5a-aba1-373c048263bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:32:59.763095 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7ca6f-7800-4e5a-aba1-373c048263bb service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:32:59.763620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7ca6f-7800-4e5a-aba1-373c048263bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:32:59.764024 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7ca6f-7800-4e5a-aba1-373c048263bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:32:59.786697 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7ca6f-7800-4e5a-aba1-373c048263bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:32:59.789544 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7ca6f-7800-4e5a-aba1-373c048263bb service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:32:59.823518 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 854/1727] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:32:59 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:00.542582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772723a-cf06-436b-aafd-6ff95fe7b21d None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:00.544056 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d772723a-cf06-436b-aafd-6ff95fe7b21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:00.573231 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772723a-cf06-436b-aafd-6ff95fe7b21d None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:00.593618 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d8409-08a0-42a2-975c-d0e874420479 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:00.594446 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d8409-08a0-42a2-975c-d0e874420479 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:00.595112 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d8409-08a0-42a2-975c-d0e874420479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:00.600815 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b98c28-8ee2-4323-bf1b-6b17d7a7444e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:00.614038 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b98c28-8ee2-4323-bf1b-6b17d7a7444e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:00.614614 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b98c28-8ee2-4323-bf1b-6b17d7a7444e admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:00.615786 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:00.620998 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b98c28-8ee2-4323-bf1b-6b17d7a7444e admin admin] PATH_INFO: `/v3/projects/c95096b8b71b4441836e289040285c19` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:00.624171 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b98c28-8ee2-4323-bf1b-6b17d7a7444e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c95096b8b71b4441836e289040285c19)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:00.627674 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b98c28-8ee2-4323-bf1b-6b17d7a7444e admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:00.711209 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-102d8409-08a0-42a2-975c-d0e874420479 None None] MFA Rules not processed for user `3170dc3323f54512bc8819c8efa86b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:00.713496 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-102d8409-08a0-42a2-975c-d0e874420479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:00.780174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-62b98c28-8ee2-4323-bf1b-6b17d7a7444e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c95096b8b71b4441836e289040285c19', 'request_id': 'req-62b98c28-8ee2-4323-bf1b-6b17d7a7444e'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:00.813798 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-62b98c28-8ee2-4323-bf1b-6b17d7a7444e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c95096b8b71b4441836e289040285c19', 'request_id': 'req-62b98c28-8ee2-4323-bf1b-6b17d7a7444e'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:00.833706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-62b98c28-8ee2-4323-bf1b-6b17d7a7444e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c95096b8b71b4441836e289040285c19', 'request_id': 'req-62b98c28-8ee2-4323-bf1b-6b17d7a7444e'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:00.895041 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-62b98c28-8ee2-4323-bf1b-6b17d7a7444e admin admin] The token cache is being invalidate because project c95096b8b71b4441836e289040285c19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:33:00.895326 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-62b98c28-8ee2-4323-bf1b-6b17d7a7444e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:00.896998 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-62b98c28-8ee2-4323-bf1b-6b17d7a7444e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:00.906033 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 874/1728] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:33:00 2022] DELETE /identity/v3/projects/c95096b8b71b4441836e289040285c19 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:00.912692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d9c0a-3fa0-48da-b6ab-9879578dda15 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:00.913290 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d9c0a-3fa0-48da-b6ab-9879578dda15 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:00.913592 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d9c0a-3fa0-48da-b6ab-9879578dda15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:00.997113 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-ee6d9c0a-3fa0-48da-b6ab-9879578dda15 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:00.999622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-ee6d9c0a-3fa0-48da-b6ab-9879578dda15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:01.011687 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 855/1729] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:00 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:01.030694 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec53832-c7df-480c-8f10-95539599cc32 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:01.030972 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec53832-c7df-480c-8f10-95539599cc32 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:01.031399 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec53832-c7df-480c-8f10-95539599cc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:01.079551 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:01.103509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7ec53832-c7df-480c-8f10-95539599cc32 None None] MFA Rules not processed for user `3170dc3323f54512bc8819c8efa86b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:01.106708 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7ec53832-c7df-480c-8f10-95539599cc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:01.218388 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 875/1730] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:00 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:01.222229 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb55da6-a482-4498-8f1a-3cbb65d29c65 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:01.222986 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb55da6-a482-4498-8f1a-3cbb65d29c65 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:01.223523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb55da6-a482-4498-8f1a-3cbb65d29c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:01.260378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 856/1731] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:01 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:01.274073 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec53832-c7df-480c-8f10-95539599cc32 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:01.275677 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-dcb55da6-a482-4498-8f1a-3cbb65d29c65 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:01.277540 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-dcb55da6-a482-4498-8f1a-3cbb65d29c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:01.277696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7ec53832-c7df-480c-8f10-95539599cc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:01.291438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 876/1732] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:01.292779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb55da6-a482-4498-8f1a-3cbb65d29c65 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:01.294744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-dcb55da6-a482-4498-8f1a-3cbb65d29c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:01.321696 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb55da6-a482-4498-8f1a-3cbb65d29c65 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:01.329498 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9930afd-00b8-4f08-993d-6d3e457fd5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:01.335555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9930afd-00b8-4f08-993d-6d3e457fd5fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:01.336069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9930afd-00b8-4f08-993d-6d3e457fd5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:01.336804 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9930afd-00b8-4f08-993d-6d3e457fd5fc admin admin] PATH_INFO: `/v3/projects/53b59a20da964076b08dfa6facd27e6e` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:01.338541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9930afd-00b8-4f08-993d-6d3e457fd5fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=53b59a20da964076b08dfa6facd27e6e)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:01.346719 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9930afd-00b8-4f08-993d-6d3e457fd5fc admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:01.442586 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f9930afd-00b8-4f08-993d-6d3e457fd5fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53b59a20da964076b08dfa6facd27e6e', 'request_id': 'req-f9930afd-00b8-4f08-993d-6d3e457fd5fc'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:01.461147 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f9930afd-00b8-4f08-993d-6d3e457fd5fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53b59a20da964076b08dfa6facd27e6e', 'request_id': 'req-f9930afd-00b8-4f08-993d-6d3e457fd5fc'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:01.472537 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f9930afd-00b8-4f08-993d-6d3e457fd5fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53b59a20da964076b08dfa6facd27e6e', 'request_id': 'req-f9930afd-00b8-4f08-993d-6d3e457fd5fc'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:01.479373 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec53832-c7df-480c-8f10-95539599cc32 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:01.522338 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f9930afd-00b8-4f08-993d-6d3e457fd5fc admin admin] The token cache is being invalidate because project 53b59a20da964076b08dfa6facd27e6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:33:01.522964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f9930afd-00b8-4f08-993d-6d3e457fd5fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:01.524134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f9930afd-00b8-4f08-993d-6d3e457fd5fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:01.526845 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 877/1733] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:33:01 2022] DELETE /identity/v3/projects/53b59a20da964076b08dfa6facd27e6e => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:01.528499 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9930afd-00b8-4f08-993d-6d3e457fd5fc admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:01.531985 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-f9930afd-00b8-4f08-993d-6d3e457fd5fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:01.584400 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:01.647713 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d933e-4e75-4ea4-ab6c-b2fe7151416a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:01.648653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d933e-4e75-4ea4-ab6c-b2fe7151416a service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:01.649672 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d933e-4e75-4ea4-ab6c-b2fe7151416a service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:01.650070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d933e-4e75-4ea4-ab6c-b2fe7151416a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:01.652820 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d76d933e-4e75-4ea4-ab6c-b2fe7151416a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:01.795793 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d933e-4e75-4ea4-ab6c-b2fe7151416a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:01.800415 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d933e-4e75-4ea4-ab6c-b2fe7151416a service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:01.820730 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 857/1734] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:01 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:01.823030 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b344ff5-ca68-4d1f-9ea8-a541a628383c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:01.823322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b344ff5-ca68-4d1f-9ea8-a541a628383c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:01.823604 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b344ff5-ca68-4d1f-9ea8-a541a628383c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:01.829021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9930afd-00b8-4f08-993d-6d3e457fd5fc admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:01.848703 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46243d0-b216-499e-adfe-44725610c2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:01.849381 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46243d0-b216-499e-adfe-44725610c2a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:01.849670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46243d0-b216-499e-adfe-44725610c2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:01.849955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46243d0-b216-499e-adfe-44725610c2a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:01.850693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46243d0-b216-499e-adfe-44725610c2a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:33:01.851003 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46243d0-b216-499e-adfe-44725610c2a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:01.864601 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46243d0-b216-499e-adfe-44725610c2a6 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:01.920700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 878/1735] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:33:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:33:01.922634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46243d0-b216-499e-adfe-44725610c2a6 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:01.927023 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-a46243d0-b216-499e-adfe-44725610c2a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:01.927203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-5b344ff5-ca68-4d1f-9ea8-a541a628383c None None] MFA Rules not processed for user `9520a49687b24281b78f08d43cb27dc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:01.928571 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5b344ff5-ca68-4d1f-9ea8-a541a628383c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:02.073711 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 858/1736] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:02.074241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46243d0-b216-499e-adfe-44725610c2a6 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:02.074669 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b344ff5-ca68-4d1f-9ea8-a541a628383c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:02.089069 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5b344ff5-ca68-4d1f-9ea8-a541a628383c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:02.091682 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b5020-3b6b-4463-9d7d-82a0e5cc006b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:02.092414 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b5020-3b6b-4463-9d7d-82a0e5cc006b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:02.092792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b5020-3b6b-4463-9d7d-82a0e5cc006b admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:02.092987 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b5020-3b6b-4463-9d7d-82a0e5cc006b admin admin] PATH_INFO: `/v3/users/9d5d847d4afd4a9fac48b4a54cf37e40` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:02.101392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b5020-3b6b-4463-9d7d-82a0e5cc006b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d5d847d4afd4a9fac48b4a54cf37e40)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:02.115882 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b5020-3b6b-4463-9d7d-82a0e5cc006b admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:02.154421 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b344ff5-ca68-4d1f-9ea8-a541a628383c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:02.176963 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d7e2f-adb4-436e-bf37-8ccb360d17af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:02.177921 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d7e2f-adb4-436e-bf37-8ccb360d17af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:02.178155 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d7e2f-adb4-436e-bf37-8ccb360d17af admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:02.178443 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d7e2f-adb4-436e-bf37-8ccb360d17af admin admin] PATH_INFO: `/v3/projects/5e3c96d4f14d49a2b01f5996b65ca61b` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:02.180805 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d7e2f-adb4-436e-bf37-8ccb360d17af admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e3c96d4f14d49a2b01f5996b65ca61b)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:02.182789 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d7e2f-adb4-436e-bf37-8ccb360d17af admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:02.231460 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:02.261470 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-718b5020-3b6b-4463-9d7d-82a0e5cc006b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d5d847d4afd4a9fac48b4a54cf37e40'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:02.262165 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-718b5020-3b6b-4463-9d7d-82a0e5cc006b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d5d847d4afd4a9fac48b4a54cf37e40'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:02.305895 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-718b5020-3b6b-4463-9d7d-82a0e5cc006b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d5d847d4afd4a9fac48b4a54cf37e40'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:02.322708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-718b5020-3b6b-4463-9d7d-82a0e5cc006b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d5d847d4afd4a9fac48b4a54cf37e40'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:02.336424 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-798d7e2f-adb4-436e-bf37-8ccb360d17af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e3c96d4f14d49a2b01f5996b65ca61b', 'request_id': 'req-798d7e2f-adb4-436e-bf37-8ccb360d17af'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:02.354221 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-718b5020-3b6b-4463-9d7d-82a0e5cc006b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d5d847d4afd4a9fac48b4a54cf37e40'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:02.356481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-798d7e2f-adb4-436e-bf37-8ccb360d17af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e3c96d4f14d49a2b01f5996b65ca61b', 'request_id': 'req-798d7e2f-adb4-436e-bf37-8ccb360d17af'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:02.363030 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 879/1737] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:33:01 2022] DELETE /identity/v3/users/9d5d847d4afd4a9fac48b4a54cf37e40 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:02.363640 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b5020-3b6b-4463-9d7d-82a0e5cc006b admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:02.369834 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:02.374401 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-718b5020-3b6b-4463-9d7d-82a0e5cc006b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:02.384886 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-798d7e2f-adb4-436e-bf37-8ccb360d17af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e3c96d4f14d49a2b01f5996b65ca61b', 'request_id': 'req-798d7e2f-adb4-436e-bf37-8ccb360d17af'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:02.415237 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-798d7e2f-adb4-436e-bf37-8ccb360d17af admin admin] The token cache is being invalidate because project 5e3c96d4f14d49a2b01f5996b65ca61b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:33:02.415731 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-798d7e2f-adb4-436e-bf37-8ccb360d17af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:02.416760 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-798d7e2f-adb4-436e-bf37-8ccb360d17af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:02.424196 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 859/1738] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:33:02 2022] DELETE /identity/v3/projects/5e3c96d4f14d49a2b01f5996b65ca61b => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:02.426051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a8d0e-a6a9-45df-9136-df07342bc123 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:02.426478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a8d0e-a6a9-45df-9136-df07342bc123 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:02.427115 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a8d0e-a6a9-45df-9136-df07342bc123 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:02.429967 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 860/1739] 10.222.0.121 () {54 vars in 917 bytes} [Thu Aug 4 10:33:02 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:33:02.432216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc33d9-e7a6-4182-a284-c5e7a7072729 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:02.432635 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc33d9-e7a6-4182-a284-c5e7a7072729 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:02.433212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc33d9-e7a6-4182-a284-c5e7a7072729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:02.468078 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-05dc33d9-e7a6-4182-a284-c5e7a7072729 None None] MFA Rules not processed for user `9520a49687b24281b78f08d43cb27dc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:02.472675 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-05dc33d9-e7a6-4182-a284-c5e7a7072729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:02.530361 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b5020-3b6b-4463-9d7d-82a0e5cc006b admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:02.546688 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e22938-b734-4710-b200-0f15fe8464e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:02.547413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e22938-b734-4710-b200-0f15fe8464e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:02.548239 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e22938-b734-4710-b200-0f15fe8464e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:02.548239 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e22938-b734-4710-b200-0f15fe8464e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:02.548741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e22938-b734-4710-b200-0f15fe8464e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:02.550957 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e22938-b734-4710-b200-0f15fe8464e8 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:02.635939 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 880/1740] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:33:02 2022] POST /identity/v3/projects => generated 372 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:02.638732 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fe687-e296-4417-a3a5-16b1a8111676 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:02.639044 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fe687-e296-4417-a3a5-16b1a8111676 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:02.639375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fe687-e296-4417-a3a5-16b1a8111676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:02.654278 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 861/1741] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:02.658481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55972a-deda-4a0b-bed2-ebc71b58014b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:02.660291 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55972a-deda-4a0b-bed2-ebc71b58014b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:02.660720 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55972a-deda-4a0b-bed2-ebc71b58014b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:02.671664 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-7f7fe687-e296-4417-a3a5-16b1a8111676 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:02.678769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-7f7fe687-e296-4417-a3a5-16b1a8111676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:02.766199 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ea55972a-deda-4a0b-bed2-ebc71b58014b None None] MFA Rules not processed for user `90c7eaf4500f4ef496ba72405522006d`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:02.767661 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ea55972a-deda-4a0b-bed2-ebc71b58014b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:02.790259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 881/1742] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:02.792776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fe687-e296-4417-a3a5-16b1a8111676 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:02.796379 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-7f7fe687-e296-4417-a3a5-16b1a8111676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:02.826481 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fe687-e296-4417-a3a5-16b1a8111676 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:02.841004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbcfc56-beda-40dc-a36c-56b59440c89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:02.841737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbcfc56-beda-40dc-a36c-56b59440c89d admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:02.841939 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbcfc56-beda-40dc-a36c-56b59440c89d admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:02.842319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbcfc56-beda-40dc-a36c-56b59440c89d admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:02.843621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbcfc56-beda-40dc-a36c-56b59440c89d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:02.846148 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbcfc56-beda-40dc-a36c-56b59440c89d admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:02.931341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 882/1743] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:33:02 2022] POST /identity/v3/users => generated 287 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:02.937579 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e5ebe-eafa-4141-9626-d0979f64991c None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:02.945174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e5ebe-eafa-4141-9626-d0979f64991c None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:02.946281 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e5ebe-eafa-4141-9626-d0979f64991c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:02.960233 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 862/1744] 10.222.0.121 () {58 vars in 1014 bytes} [Thu Aug 4 10:33:02 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:02.962365 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55972a-deda-4a0b-bed2-ebc71b58014b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:02.993990 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55972a-deda-4a0b-bed2-ebc71b58014b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:03.002914 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-4a3e5ebe-eafa-4141-9626-d0979f64991c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:03.006209 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-4a3e5ebe-eafa-4141-9626-d0979f64991c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:03.008733 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b6292-333e-4e9b-9ef6-1a914758c7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:03.009415 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b6292-333e-4e9b-9ef6-1a914758c7e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:03.009706 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b6292-333e-4e9b-9ef6-1a914758c7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:03.010055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b6292-333e-4e9b-9ef6-1a914758c7e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:03.011913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b6292-333e-4e9b-9ef6-1a914758c7e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:03.016668 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b6292-333e-4e9b-9ef6-1a914758c7e0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:03.022491 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 883/1745] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:03.027273 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e5ebe-eafa-4141-9626-d0979f64991c None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:03.048669 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 863/1746] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:02 2022] GET /identity/v3/roles => generated 1427 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:03.051155 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b6292-333e-4e9b-9ef6-1a914758c7e0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:03.051321 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e5ebe-eafa-4141-9626-d0979f64991c None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:03.061553 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6c4e3-b193-4e94-ad70-749dbc3f5c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:03.062645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6c4e3-b193-4e94-ad70-749dbc3f5c95 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:03.063079 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6c4e3-b193-4e94-ad70-749dbc3f5c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:03.063753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6c4e3-b193-4e94-ad70-749dbc3f5c95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:03.064664 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e6c4e3-b193-4e94-ad70-749dbc3f5c95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:33:03.065088 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e6c4e3-b193-4e94-ad70-749dbc3f5c95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:03.067893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e6c4e3-b193-4e94-ad70-749dbc3f5c95 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:03.073944 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b6292-333e-4e9b-9ef6-1a914758c7e0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:03.087894 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7daa06-ea1f-4f51-9cf5-6be37bb5e02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:03.088809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7daa06-ea1f-4f51-9cf5-6be37bb5e02d admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:03.089106 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7daa06-ea1f-4f51-9cf5-6be37bb5e02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:03.089391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7daa06-ea1f-4f51-9cf5-6be37bb5e02d admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:03.090117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7daa06-ea1f-4f51-9cf5-6be37bb5e02d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:03.097305 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7daa06-ea1f-4f51-9cf5-6be37bb5e02d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:03.122670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 884/1747] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:33:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:33:03.124167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 864/1748] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:03 2022] GET /identity/v3/roles => generated 1427 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:03.128782 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6c4e3-b193-4e94-ad70-749dbc3f5c95 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:03.129426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7daa06-ea1f-4f51-9cf5-6be37bb5e02d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:03.140378 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:03.161798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7daa06-ea1f-4f51-9cf5-6be37bb5e02d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:03.167706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6c4e3-b193-4e94-ad70-749dbc3f5c95 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:03.187204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c694e-f602-40be-b14b-e0df14ef0f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:03.188222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c694e-f602-40be-b14b-e0df14ef0f3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:03.188600 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c694e-f602-40be-b14b-e0df14ef0f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:03.188988 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c694e-f602-40be-b14b-e0df14ef0f3a admin admin] PATH_INFO: `/v3/projects/9f9794c4379f4aa5a930a8dd20be7628/users/0a08d4bc8dde46748c82dd3a5409f230/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:03.189650 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e749c70-1c79-4f76-a9b6-6d1c3e598f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:03.190949 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e749c70-1c79-4f76-a9b6-6d1c3e598f87 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:03.191287 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e749c70-1c79-4f76-a9b6-6d1c3e598f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:03.193018 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e749c70-1c79-4f76-a9b6-6d1c3e598f87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:03.195075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e749c70-1c79-4f76-a9b6-6d1c3e598f87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:03.203796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e749c70-1c79-4f76-a9b6-6d1c3e598f87 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:03.235240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c694e-f602-40be-b14b-e0df14ef0f3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f9794c4379f4aa5a930a8dd20be7628, user_id=0a08d4bc8dde46748c82dd3a5409f230, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:03.239245 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c694e-f602-40be-b14b-e0df14ef0f3a admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:03.277867 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 885/1749] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:33:03 2022] PUT /identity/v3/projects/9f9794c4379f4aa5a930a8dd20be7628/users/0a08d4bc8dde46748c82dd3a5409f230/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:03.277867 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c694e-f602-40be-b14b-e0df14ef0f3a admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:03.281377 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-9d6c694e-f602-40be-b14b-e0df14ef0f3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:03.281515 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 865/1750] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:33:03 2022] POST /identity/v3/projects => generated 378 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:03.282307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67da071-e198-40cc-a23b-8f2eb3ad032d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:03.282602 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67da071-e198-40cc-a23b-8f2eb3ad032d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:03.282883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67da071-e198-40cc-a23b-8f2eb3ad032d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:03.347580 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e67da071-e198-40cc-a23b-8f2eb3ad032d None None] MFA Rules not processed for user `a0d4f49c68d3442f89c33b3f7f9fd9aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:03.349312 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e67da071-e198-40cc-a23b-8f2eb3ad032d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:03.428261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:03.456829 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c694e-f602-40be-b14b-e0df14ef0f3a admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:03.529327 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 866/1751] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:03 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:03.530853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3c038-9253-4bc5-be7e-2f16bf50763d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:03.531151 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3c038-9253-4bc5-be7e-2f16bf50763d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:03.531438 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3c038-9253-4bc5-be7e-2f16bf50763d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:03.562392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd165b-17c5-415f-ad09-6a2b15541e1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:03.563751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd165b-17c5-415f-ad09-6a2b15541e1c service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:03.564094 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd165b-17c5-415f-ad09-6a2b15541e1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:03.564380 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd165b-17c5-415f-ad09-6a2b15541e1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:03.575536 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd165b-17c5-415f-ad09-6a2b15541e1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:03.577435 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd165b-17c5-415f-ad09-6a2b15541e1c service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:03.597509 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 886/1752] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:03 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:03.598843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd165b-17c5-415f-ad09-6a2b15541e1c service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:03.611810 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-eab3c038-9253-4bc5-be7e-2f16bf50763d None None] MFA Rules not processed for user `0a08d4bc8dde46748c82dd3a5409f230`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:03.620006 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-eab3c038-9253-4bc5-be7e-2f16bf50763d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:03.773865 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 867/1753] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:03.779981 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a51cb7-2edf-4fe8-b095-a685e00b6cb0 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:03.780357 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a51cb7-2edf-4fe8-b095-a685e00b6cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:03.783030 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a51cb7-2edf-4fe8-b095-a685e00b6cb0 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:03.788466 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 868/1754] 10.222.0.121 () {54 vars in 917 bytes} [Thu Aug 4 10:33:03 2022] GET /identity => generated 271 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:33:03.789607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a51cb7-2edf-4fe8-b095-a685e00b6cb0 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:03.805369 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c7a51cb7-2edf-4fe8-b095-a685e00b6cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:03.818705 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd165b-17c5-415f-ad09-6a2b15541e1c service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:03.831222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc1871-f5c1-4b93-bd5b-a6305203f0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:03.831893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc1871-f5c1-4b93-bd5b-a6305203f0eb admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:03.832199 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc1871-f5c1-4b93-bd5b-a6305203f0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:03.832476 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc1871-f5c1-4b93-bd5b-a6305203f0eb admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:03.833429 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbc1871-f5c1-4b93-bd5b-a6305203f0eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:03.837771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbc1871-f5c1-4b93-bd5b-a6305203f0eb admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:03.888283 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 887/1755] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:33:03 2022] POST /identity/v3/users => generated 290 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:03.890612 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94c208-917c-458c-a30f-2af74be29628 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:03.891161 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94c208-917c-458c-a30f-2af74be29628 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:03.892300 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94c208-917c-458c-a30f-2af74be29628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:03.935681 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-bb94c208-917c-458c-a30f-2af74be29628 None None] MFA Rules not processed for user `90c7eaf4500f4ef496ba72405522006d`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:03.936997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-bb94c208-917c-458c-a30f-2af74be29628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:03.952814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a51cb7-2edf-4fe8-b095-a685e00b6cb0 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:03.956181 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 888/1756] 10.222.0.121 () {58 vars in 1014 bytes} [Thu Aug 4 10:33:03 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:03.957106 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb94c208-917c-458c-a30f-2af74be29628 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:03.962173 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e4e0c-2ef7-4f27-b8e5-e002441dc23a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:03.962849 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e4e0c-2ef7-4f27-b8e5-e002441dc23a service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:03.963135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e4e0c-2ef7-4f27-b8e5-e002441dc23a service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:03.963414 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e4e0c-2ef7-4f27-b8e5-e002441dc23a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:03.970205 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f36e4e0c-2ef7-4f27-b8e5-e002441dc23a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:03.977509 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb94c208-917c-458c-a30f-2af74be29628 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:03.985953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fae7d-5538-434b-b8e9-b75f5e4865e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:03.997901 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fae7d-5538-434b-b8e9-b75f5e4865e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:03.998299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fae7d-5538-434b-b8e9-b75f5e4865e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:03.998570 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fae7d-5538-434b-b8e9-b75f5e4865e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:03.999349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298fae7d-5538-434b-b8e9-b75f5e4865e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:04.002888 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298fae7d-5538-434b-b8e9-b75f5e4865e0 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:04.019983 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 889/1757] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:03 2022] GET /identity/v3/roles => generated 1427 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:04.020458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fae7d-5538-434b-b8e9-b75f5e4865e0 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:04.023318 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-298fae7d-5538-434b-b8e9-b75f5e4865e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:04.047681 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fae7d-5538-434b-b8e9-b75f5e4865e0 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:04.056056 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a97be-0343-4f48-b15a-93092283d62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:04.057711 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a97be-0343-4f48-b15a-93092283d62d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:04.058133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a97be-0343-4f48-b15a-93092283d62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:04.058542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a97be-0343-4f48-b15a-93092283d62d admin admin] PATH_INFO: `/v3/projects/2c7d8b766dd644cbb1aa6b0af0ff5895` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:04.060127 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a97be-0343-4f48-b15a-93092283d62d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c7d8b766dd644cbb1aa6b0af0ff5895)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:04.064477 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a97be-0343-4f48-b15a-93092283d62d admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:04.068086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:04.098244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e4e0c-2ef7-4f27-b8e5-e002441dc23a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:04.100007 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e4e0c-2ef7-4f27-b8e5-e002441dc23a service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:04.112716 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 869/1758] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:04.113446 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e4e0c-2ef7-4f27-b8e5-e002441dc23a service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:04.140160 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e4e0c-2ef7-4f27-b8e5-e002441dc23a service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:04.148906 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5f174-9ef4-4994-a223-6b63012009a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:04.149533 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5f174-9ef4-4994-a223-6b63012009a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:04.149822 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5f174-9ef4-4994-a223-6b63012009a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:04.150397 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5f174-9ef4-4994-a223-6b63012009a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:04.151776 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd5f174-9ef4-4994-a223-6b63012009a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:04.156440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd5f174-9ef4-4994-a223-6b63012009a5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:04.167137 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:04.180796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 870/1759] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:04 2022] GET /identity/v3/roles => generated 1427 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:04.182988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5f174-9ef4-4994-a223-6b63012009a5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:04.192527 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-856a97be-0343-4f48-b15a-93092283d62d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c7d8b766dd644cbb1aa6b0af0ff5895', 'request_id': 'req-856a97be-0343-4f48-b15a-93092283d62d'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:04.204164 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5f174-9ef4-4994-a223-6b63012009a5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:04.206835 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-856a97be-0343-4f48-b15a-93092283d62d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c7d8b766dd644cbb1aa6b0af0ff5895', 'request_id': 'req-856a97be-0343-4f48-b15a-93092283d62d'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:04.214615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-856a97be-0343-4f48-b15a-93092283d62d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c7d8b766dd644cbb1aa6b0af0ff5895', 'request_id': 'req-856a97be-0343-4f48-b15a-93092283d62d'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:04.233671 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-856a97be-0343-4f48-b15a-93092283d62d admin admin] The token cache is being invalidate because project 2c7d8b766dd644cbb1aa6b0af0ff5895 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:33:04.234085 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-856a97be-0343-4f48-b15a-93092283d62d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:04.235767 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-856a97be-0343-4f48-b15a-93092283d62d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:04.240168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 890/1760] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:33:04 2022] DELETE /identity/v3/projects/2c7d8b766dd644cbb1aa6b0af0ff5895 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:04.240168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a97be-0343-4f48-b15a-93092283d62d admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:04.246009 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-856a97be-0343-4f48-b15a-93092283d62d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:04.314934 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e68f5a-5d2c-476f-ba12-33c018ae9f26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:04.316176 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e68f5a-5d2c-476f-ba12-33c018ae9f26 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:04.316723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e68f5a-5d2c-476f-ba12-33c018ae9f26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:04.317612 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e68f5a-5d2c-476f-ba12-33c018ae9f26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:04.322488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-66e68f5a-5d2c-476f-ba12-33c018ae9f26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:04.342262 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e68f5a-5d2c-476f-ba12-33c018ae9f26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:04.344426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e68f5a-5d2c-476f-ba12-33c018ae9f26 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:04.361720 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 871/1761] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:04 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:04.364341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87866e6e-fb20-4912-a189-eb1ab9fb0e92 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:04.364746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87866e6e-fb20-4912-a189-eb1ab9fb0e92 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:04.365283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87866e6e-fb20-4912-a189-eb1ab9fb0e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:04.427708 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-87866e6e-fb20-4912-a189-eb1ab9fb0e92 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:04.433953 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-87866e6e-fb20-4912-a189-eb1ab9fb0e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:04.488552 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a97be-0343-4f48-b15a-93092283d62d admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:04.503456 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f4add-2ee1-4da2-924c-c5a3b2e5fc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:04.504242 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f4add-2ee1-4da2-924c-c5a3b2e5fc84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:04.504537 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f4add-2ee1-4da2-924c-c5a3b2e5fc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:04.505052 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f4add-2ee1-4da2-924c-c5a3b2e5fc84 admin admin] PATH_INFO: `/v3/projects/cc2a4e4a15c741978625363534446549/users/9ae85e53631f4de8b254cefb428ee229/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:04.556910 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f4add-2ee1-4da2-924c-c5a3b2e5fc84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc2a4e4a15c741978625363534446549, user_id=9ae85e53631f4de8b254cefb428ee229, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:04.570511 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f4add-2ee1-4da2-924c-c5a3b2e5fc84 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:04.614653 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 891/1762] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:33:04 2022] PUT /identity/v3/projects/cc2a4e4a15c741978625363534446549/users/9ae85e53631f4de8b254cefb428ee229/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:04.629241 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 872/1763] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:04.630296 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99476c-3be9-44cb-b780-e9e744e70314 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:04.632492 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99476c-3be9-44cb-b780-e9e744e70314 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:04.635698 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98930d27-fcbd-4710-a9d5-88ff0948bebd None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:04.636017 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98930d27-fcbd-4710-a9d5-88ff0948bebd None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:04.636270 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99476c-3be9-44cb-b780-e9e744e70314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:04.636368 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98930d27-fcbd-4710-a9d5-88ff0948bebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:04.741359 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-4d99476c-3be9-44cb-b780-e9e744e70314 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:04.743317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-4d99476c-3be9-44cb-b780-e9e744e70314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:04.785938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-98930d27-fcbd-4710-a9d5-88ff0948bebd None None] MFA Rules not processed for user `9ae85e53631f4de8b254cefb428ee229`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:04.787811 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-98930d27-fcbd-4710-a9d5-88ff0948bebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:04.914157 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 892/1764] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:04.919582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612fb19-d25a-4669-9738-bcdef4cf5ea6 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:04.919914 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612fb19-d25a-4669-9738-bcdef4cf5ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:04.920176 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612fb19-d25a-4669-9738-bcdef4cf5ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:04.945124 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 873/1765] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:04.946242 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98930d27-fcbd-4710-a9d5-88ff0948bebd None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:04.948163 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-98930d27-fcbd-4710-a9d5-88ff0948bebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:04.959411 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-a612fb19-d25a-4669-9738-bcdef4cf5ea6 None None] MFA Rules not processed for user `41a38b5af1654f44bc6cb5832c2e8e87`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:04.960759 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-a612fb19-d25a-4669-9738-bcdef4cf5ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:04.991468 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:05.013270 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:05.085601 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 893/1766] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:04 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:05.088321 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612fb19-d25a-4669-9738-bcdef4cf5ea6 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:05.117330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612fb19-d25a-4669-9738-bcdef4cf5ea6 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:05.130930 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09473be1-27c1-45a1-ab58-5d93a4f62bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:05.131640 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09473be1-27c1-45a1-ab58-5d93a4f62bcc admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:05.131640 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09473be1-27c1-45a1-ab58-5d93a4f62bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:05.132850 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09473be1-27c1-45a1-ab58-5d93a4f62bcc admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:05.132850 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09473be1-27c1-45a1-ab58-5d93a4f62bcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:33:05.133058 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09473be1-27c1-45a1-ab58-5d93a4f62bcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:05.139511 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09473be1-27c1-45a1-ab58-5d93a4f62bcc admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:05.144677 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98930d27-fcbd-4710-a9d5-88ff0948bebd None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:05.159578 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be8765-20c8-4b44-a1a8-e1bd225a4515 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:05.160301 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be8765-20c8-4b44-a1a8-e1bd225a4515 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:05.160566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be8765-20c8-4b44-a1a8-e1bd225a4515 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:05.160869 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be8765-20c8-4b44-a1a8-e1bd225a4515 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:05.166915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c9be8765-20c8-4b44-a1a8-e1bd225a4515 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:05.187331 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 894/1767] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:33:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:33:05.187331 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6127942-8f27-4403-8c4b-f6842d5c136a None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:05.187659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6127942-8f27-4403-8c4b-f6842d5c136a None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:05.188007 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6127942-8f27-4403-8c4b-f6842d5c136a None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:05.193208 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 895/1768] 10.222.0.121 () {54 vars in 966 bytes} [Thu Aug 4 10:33:05 2022] GET /identity => generated 271 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:33:05.195472 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6127942-8f27-4403-8c4b-f6842d5c136a None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:05.224128 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6127942-8f27-4403-8c4b-f6842d5c136a None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:05.235193 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b13d2-4d4c-446a-9e63-b983258aad7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:05.236081 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b13d2-4d4c-446a-9e63-b983258aad7d admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:05.236480 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b13d2-4d4c-446a-9e63-b983258aad7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:05.236862 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b13d2-4d4c-446a-9e63-b983258aad7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:05.240976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b13d2-4d4c-446a-9e63-b983258aad7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:05.245887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b13d2-4d4c-446a-9e63-b983258aad7d admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:05.279322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be8765-20c8-4b44-a1a8-e1bd225a4515 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:05.280985 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be8765-20c8-4b44-a1a8-e1bd225a4515 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:05.295058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 874/1769] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:04 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:05.295058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be8765-20c8-4b44-a1a8-e1bd225a4515 service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:05.307534 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be8765-20c8-4b44-a1a8-e1bd225a4515 service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:05.312404 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 896/1770] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:33:05 2022] POST /identity/v3/projects => generated 376 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:05.316395 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12529495-e3ad-416e-a918-eefdca9c1df0 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:05.317014 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12529495-e3ad-416e-a918-eefdca9c1df0 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:05.317402 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12529495-e3ad-416e-a918-eefdca9c1df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:05.359275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-12529495-e3ad-416e-a918-eefdca9c1df0 None None] MFA Rules not processed for user `01c7cfbe911c4378857848966f4f8d98`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:05.361695 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-12529495-e3ad-416e-a918-eefdca9c1df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:05.422227 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7a91a-5119-427f-a056-95a39f69a803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:05.422946 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7a91a-5119-427f-a056-95a39f69a803 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:05.423313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7a91a-5119-427f-a056-95a39f69a803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:05.423620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7a91a-5119-427f-a056-95a39f69a803 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:05.424369 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7a91a-5119-427f-a056-95a39f69a803 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:05.426352 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7a91a-5119-427f-a056-95a39f69a803 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:05.449151 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 897/1771] 10.222.0.121 () {58 vars in 1063 bytes} [Thu Aug 4 10:33:05 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:05.450504 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12529495-e3ad-416e-a918-eefdca9c1df0 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:05.452157 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-12529495-e3ad-416e-a918-eefdca9c1df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:05.464013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 875/1772] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:33:05 2022] POST /identity/v3/projects => generated 370 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:05.465277 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7a91a-5119-427f-a056-95a39f69a803 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:05.535826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:05.547501 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7a91a-5119-427f-a056-95a39f69a803 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:05.561422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12529495-e3ad-416e-a918-eefdca9c1df0 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:05.562645 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fbf7fe-52be-4080-80e5-265963f550e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:05.563401 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbf7fe-52be-4080-80e5-265963f550e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:05.563866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbf7fe-52be-4080-80e5-265963f550e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:05.564178 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbf7fe-52be-4080-80e5-265963f550e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:05.564875 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fbf7fe-52be-4080-80e5-265963f550e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:05.566843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fbf7fe-52be-4080-80e5-265963f550e9 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:05.585266 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:05.608105 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 876/1773] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:33:05 2022] POST /identity/v3/users => generated 295 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:05.608105 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fbf7fe-52be-4080-80e5-265963f550e9 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:05.611960 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b7fbf7fe-52be-4080-80e5-265963f550e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:05.652681 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68397322-c7e3-4969-9a85-ca6024f5a087 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:05.653589 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68397322-c7e3-4969-9a85-ca6024f5a087 service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:05.653808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68397322-c7e3-4969-9a85-ca6024f5a087 service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:05.654036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68397322-c7e3-4969-9a85-ca6024f5a087 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:05.656159 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-68397322-c7e3-4969-9a85-ca6024f5a087 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:05.693574 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fbf7fe-52be-4080-80e5-265963f550e9 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:05.699710 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b582b4f-6748-4019-aa19-69f5dcefcfc3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c7cfbe911c4378857848966f4f8d98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:05.701213 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b582b4f-6748-4019-aa19-69f5dcefcfc3 service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:05.701213 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b582b4f-6748-4019-aa19-69f5dcefcfc3 service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:05.701487 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b582b4f-6748-4019-aa19-69f5dcefcfc3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:05.703456 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0b582b4f-6748-4019-aa19-69f5dcefcfc3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:05.717187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b582b4f-6748-4019-aa19-69f5dcefcfc3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:05.718887 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b582b4f-6748-4019-aa19-69f5dcefcfc3 service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:05.728790 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68397322-c7e3-4969-9a85-ca6024f5a087 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:05.730537 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 877/1774] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:05 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:05.731186 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68397322-c7e3-4969-9a85-ca6024f5a087 service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:05.733259 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b582b4f-6748-4019-aa19-69f5dcefcfc3 service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:05.747517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 898/1775] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:05.749111 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68397322-c7e3-4969-9a85-ca6024f5a087 service nova] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:05.755212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b582b4f-6748-4019-aa19-69f5dcefcfc3 service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:05.765137 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38aa9ed-e5c1-43f7-b7eb-d8a51961a458 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c7cfbe911c4378857848966f4f8d98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:05.766626 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38aa9ed-e5c1-43f7-b7eb-d8a51961a458 service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:05.767300 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38aa9ed-e5c1-43f7-b7eb-d8a51961a458 service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:05.767495 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68397322-c7e3-4969-9a85-ca6024f5a087 service nova] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:05.768181 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38aa9ed-e5c1-43f7-b7eb-d8a51961a458 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:05.775399 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5605be-ad4b-4c2d-bec4-d3bfbe5ba2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:05.776471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5605be-ad4b-4c2d-bec4-d3bfbe5ba2fe admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:05.776964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5605be-ad4b-4c2d-bec4-d3bfbe5ba2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:05.777906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5605be-ad4b-4c2d-bec4-d3bfbe5ba2fe admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:05.779036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5605be-ad4b-4c2d-bec4-d3bfbe5ba2fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:05.782806 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5605be-ad4b-4c2d-bec4-d3bfbe5ba2fe admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:05.792577 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38aa9ed-e5c1-43f7-b7eb-d8a51961a458 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:05.795904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38aa9ed-e5c1-43f7-b7eb-d8a51961a458 service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:05.833957 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 899/1777] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:33:05 2022] POST /identity/v3/users => generated 286 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:05.834411 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 878/1777] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:05 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:05.835723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5605be-ad4b-4c2d-bec4-d3bfbe5ba2fe admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:05.839332 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38aa9ed-e5c1-43f7-b7eb-d8a51961a458 service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:05.880206 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38aa9ed-e5c1-43f7-b7eb-d8a51961a458 service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:05.903483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5605be-ad4b-4c2d-bec4-d3bfbe5ba2fe admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:05.916615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554937d0-e626-48b2-b098-4aa6b0f92f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:05.919000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ce294-eede-4c0f-b400-cfd6a671b21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:05.930014 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554937d0-e626-48b2-b098-4aa6b0f92f11 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:05.931412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ce294-eede-4c0f-b400-cfd6a671b21f admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:05.931888 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ce294-eede-4c0f-b400-cfd6a671b21f admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:05.932514 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ce294-eede-4c0f-b400-cfd6a671b21f admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:05.932861 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554937d0-e626-48b2-b098-4aa6b0f92f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:05.934054 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554937d0-e626-48b2-b098-4aa6b0f92f11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:05.934634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ce294-eede-4c0f-b400-cfd6a671b21f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:05.937756 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ce294-eede-4c0f-b400-cfd6a671b21f admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:05.946075 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554937d0-e626-48b2-b098-4aa6b0f92f11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:05.968138 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554937d0-e626-48b2-b098-4aa6b0f92f11 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:05.974739 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 900/1778] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:05 2022] GET /identity/v3/roles => generated 1427 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:05.976353 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ce294-eede-4c0f-b400-cfd6a671b21f admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:06.018033 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ce294-eede-4c0f-b400-cfd6a671b21f admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:06.044144 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d277ad49-0191-49de-aca1-29780fc82ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:06.044375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277ad49-0191-49de-aca1-29780fc82ca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:06.047638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277ad49-0191-49de-aca1-29780fc82ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:06.054323 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 879/1779] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:05 2022] GET /identity/v3/roles => generated 1427 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:06.059339 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277ad49-0191-49de-aca1-29780fc82ca2 admin admin] PATH_INFO: `/v3/projects/89cd83fc7f994298a562b2a66b7e4f5e/users/143954d97cfa491cbd467f346e51e184/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:06.060195 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554937d0-e626-48b2-b098-4aa6b0f92f11 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:06.098402 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554937d0-e626-48b2-b098-4aa6b0f92f11 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:06.128667 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d277ad49-0191-49de-aca1-29780fc82ca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89cd83fc7f994298a562b2a66b7e4f5e, user_id=143954d97cfa491cbd467f346e51e184, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:06.133573 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08669fc0-e512-463a-8513-08e00b951aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:06.134401 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08669fc0-e512-463a-8513-08e00b951aae admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:06.134930 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08669fc0-e512-463a-8513-08e00b951aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:06.135138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d277ad49-0191-49de-aca1-29780fc82ca2 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:06.137085 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08669fc0-e512-463a-8513-08e00b951aae admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:06.138954 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08669fc0-e512-463a-8513-08e00b951aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:06.145151 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08669fc0-e512-463a-8513-08e00b951aae admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:06.159154 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 901/1780] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:33:05 2022] PUT /identity/v3/projects/89cd83fc7f994298a562b2a66b7e4f5e/users/143954d97cfa491cbd467f346e51e184/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:06.161223 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d277ad49-0191-49de-aca1-29780fc82ca2 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:06.165055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d277ad49-0191-49de-aca1-29780fc82ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:06.178361 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 880/1781] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:06 2022] GET /identity/v3/roles => generated 1427 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:06.189292 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08669fc0-e512-463a-8513-08e00b951aae admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:06.244186 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08669fc0-e512-463a-8513-08e00b951aae admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:06.342937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d277ad49-0191-49de-aca1-29780fc82ca2 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:06.360177 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527d7b2-dfa8-4b89-87a6-f8c200dbc306 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:06.362300 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527d7b2-dfa8-4b89-87a6-f8c200dbc306 service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:06.363241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527d7b2-dfa8-4b89-87a6-f8c200dbc306 service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:06.364385 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527d7b2-dfa8-4b89-87a6-f8c200dbc306 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:06.367537 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c527d7b2-dfa8-4b89-87a6-f8c200dbc306 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:06.419581 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:06.472718 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197b50e-d90c-485c-8d8c-3fc031fcf55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:06.479007 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197b50e-d90c-485c-8d8c-3fc031fcf55e admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:06.481746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197b50e-d90c-485c-8d8c-3fc031fcf55e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:06.481746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197b50e-d90c-485c-8d8c-3fc031fcf55e admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:06.485367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197b50e-d90c-485c-8d8c-3fc031fcf55e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:06.502703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197b50e-d90c-485c-8d8c-3fc031fcf55e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:06.554272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 881/1782] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:06 2022] GET /identity/v3/roles => generated 1427 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:06.554272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197b50e-d90c-485c-8d8c-3fc031fcf55e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:06.590950 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197b50e-d90c-485c-8d8c-3fc031fcf55e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:06.603286 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7f8a1-b5dd-4c5f-9847-c86f70bb984b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:06.604722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7f8a1-b5dd-4c5f-9847-c86f70bb984b admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:06.606322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7f8a1-b5dd-4c5f-9847-c86f70bb984b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:06.606750 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7f8a1-b5dd-4c5f-9847-c86f70bb984b admin admin] PATH_INFO: `/v3/projects/7514c7229ff0406aa761c50cefd3492c/users/15d4ade3a53540388df740f14dd67631/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:06.634605 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527d7b2-dfa8-4b89-87a6-f8c200dbc306 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:06.640935 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527d7b2-dfa8-4b89-87a6-f8c200dbc306 service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:06.666161 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 902/1783] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:06 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:06.670498 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527d7b2-dfa8-4b89-87a6-f8c200dbc306 service cinder] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:06.675613 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b7f8a1-b5dd-4c5f-9847-c86f70bb984b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7514c7229ff0406aa761c50cefd3492c, user_id=15d4ade3a53540388df740f14dd67631, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:06.681527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b7f8a1-b5dd-4c5f-9847-c86f70bb984b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:06.692026 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527d7b2-dfa8-4b89-87a6-f8c200dbc306 service cinder] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:06.705496 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470a737-7acc-4c16-93bc-55a55f53e623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:06.707153 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470a737-7acc-4c16-93bc-55a55f53e623 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:06.708712 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470a737-7acc-4c16-93bc-55a55f53e623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:06.709364 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470a737-7acc-4c16-93bc-55a55f53e623 admin admin] PATH_INFO: `/v3/projects/89cd83fc7f994298a562b2a66b7e4f5e/users/143954d97cfa491cbd467f346e51e184/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:06.710647 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 882/1784] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:33:06 2022] PUT /identity/v3/projects/7514c7229ff0406aa761c50cefd3492c/users/15d4ade3a53540388df740f14dd67631/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:06.714551 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470a737-7acc-4c16-93bc-55a55f53e623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89cd83fc7f994298a562b2a66b7e4f5e, user_id=143954d97cfa491cbd467f346e51e184, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:06.722256 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1b802-8531-44b0-a3b3-efa7afa397fa None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:06.722895 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1b802-8531-44b0-a3b3-efa7afa397fa None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:06.723385 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1b802-8531-44b0-a3b3-efa7afa397fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:06.757192 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470a737-7acc-4c16-93bc-55a55f53e623 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:06.808331 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:06.847782 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 903/1785] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:33:06 2022] PUT /identity/v3/projects/89cd83fc7f994298a562b2a66b7e4f5e/users/143954d97cfa491cbd467f346e51e184/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:06.858548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-fdc1b802-8531-44b0-a3b3-efa7afa397fa None None] MFA Rules not processed for user `15d4ade3a53540388df740f14dd67631`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:06.859064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126c808-244d-4366-80a4-ba059b29507d None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:06.859849 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fdc1b802-8531-44b0-a3b3-efa7afa397fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:06.864133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126c808-244d-4366-80a4-ba059b29507d None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:06.864567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126c808-244d-4366-80a4-ba059b29507d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:07.070059 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-9126c808-244d-4366-80a4-ba059b29507d None None] MFA Rules not processed for user `143954d97cfa491cbd467f346e51e184`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:07.074874 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-9126c808-244d-4366-80a4-ba059b29507d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:07.241751 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 883/1786] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 521 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:07.253000 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bdfa1-3558-40b1-9262-43d82678f51e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:07.253310 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bdfa1-3558-40b1-9262-43d82678f51e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:07.253586 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bdfa1-3558-40b1-9262-43d82678f51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:07.299112 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-551bdfa1-3558-40b1-9262-43d82678f51e None None] MFA Rules not processed for user `3170dc3323f54512bc8819c8efa86b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:07.300789 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-551bdfa1-3558-40b1-9262-43d82678f51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:07.424576 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 904/1787] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:06 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 572 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:07.424576 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9126c808-244d-4366-80a4-ba059b29507d None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:07.456514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 884/1788] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:07 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:07.457910 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551bdfa1-3558-40b1-9262-43d82678f51e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:07.466044 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-551bdfa1-3558-40b1-9262-43d82678f51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:07.637095 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551bdfa1-3558-40b1-9262-43d82678f51e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:07.649463 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103213d-d85c-4827-be4c-b96755e607e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:07.650719 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103213d-d85c-4827-be4c-b96755e607e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:07.657755 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103213d-d85c-4827-be4c-b96755e607e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:07.658085 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103213d-d85c-4827-be4c-b96755e607e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:07.673194 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9126c808-244d-4366-80a4-ba059b29507d None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:07.692040 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d92a25-5f3b-45ba-8e67-af0ca33d0314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:07.692708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d92a25-5f3b-45ba-8e67-af0ca33d0314 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:07.692993 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d92a25-5f3b-45ba-8e67-af0ca33d0314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:07.693275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d92a25-5f3b-45ba-8e67-af0ca33d0314 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:07.697278 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d92a25-5f3b-45ba-8e67-af0ca33d0314 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:07.699269 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d92a25-5f3b-45ba-8e67-af0ca33d0314 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:07.810464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 905/1789] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:33:07 2022] POST /identity/v3/projects => generated 370 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:07.815132 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d92a25-5f3b-45ba-8e67-af0ca33d0314 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:07.868023 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:07.896451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103213d-d85c-4827-be4c-b96755e607e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:07.903609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103213d-d85c-4827-be4c-b96755e607e9 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:07.918668 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:07.941085 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 885/1790] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:07 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:08.056770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d92a25-5f3b-45ba-8e67-af0ca33d0314 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:08.075856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb048b-eb79-4b85-813d-cd08d137b309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:08.081970 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb048b-eb79-4b85-813d-cd08d137b309 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:08.082520 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb048b-eb79-4b85-813d-cd08d137b309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:08.084187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb048b-eb79-4b85-813d-cd08d137b309 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:08.085101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb048b-eb79-4b85-813d-cd08d137b309 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:08.094966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb048b-eb79-4b85-813d-cd08d137b309 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:08.167919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 906/1791] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:33:07 2022] POST /identity/v3/users => generated 292 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:08.172449 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103213d-d85c-4827-be4c-b96755e607e9 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:08.189114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103213d-d85c-4827-be4c-b96755e607e9 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:08.203226 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f30b5-582c-435a-9df3-6488ea9c1fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:08.204034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f30b5-582c-435a-9df3-6488ea9c1fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:08.204323 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f30b5-582c-435a-9df3-6488ea9c1fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:08.204599 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f30b5-582c-435a-9df3-6488ea9c1fc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:08.205353 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f30b5-582c-435a-9df3-6488ea9c1fc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:08.207089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f30b5-582c-435a-9df3-6488ea9c1fc0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:08.237085 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 886/1792] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:08 2022] GET /identity/v3/roles => generated 1427 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:08.242626 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb048b-eb79-4b85-813d-cd08d137b309 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:08.281024 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb048b-eb79-4b85-813d-cd08d137b309 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:08.299558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34de271-fa69-4a26-ba44-b6cd9d586b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:08.300696 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34de271-fa69-4a26-ba44-b6cd9d586b56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:08.302318 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34de271-fa69-4a26-ba44-b6cd9d586b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:08.303173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34de271-fa69-4a26-ba44-b6cd9d586b56 admin admin] PATH_INFO: `/v3/projects/5dcf994c0f9c4575b4f24faf935e4f45/users/dc6a52d1e93e4951aa2b48e7510173d4/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:08.347849 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34de271-fa69-4a26-ba44-b6cd9d586b56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dcf994c0f9c4575b4f24faf935e4f45, user_id=dc6a52d1e93e4951aa2b48e7510173d4, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:08.363113 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34de271-fa69-4a26-ba44-b6cd9d586b56 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:08.407695 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 907/1793] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:33:08 2022] PUT /identity/v3/projects/5dcf994c0f9c4575b4f24faf935e4f45/users/dc6a52d1e93e4951aa2b48e7510173d4/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:08.411419 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f30b5-582c-435a-9df3-6488ea9c1fc0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:08.435448 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f30b5-582c-435a-9df3-6488ea9c1fc0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:08.595049 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00915139-4cdc-452d-b541-0f2aa70c8bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:08.596981 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00915139-4cdc-452d-b541-0f2aa70c8bbb admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:08.600538 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00915139-4cdc-452d-b541-0f2aa70c8bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:08.600954 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00915139-4cdc-452d-b541-0f2aa70c8bbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:08.602175 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00915139-4cdc-452d-b541-0f2aa70c8bbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:08.604439 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00915139-4cdc-452d-b541-0f2aa70c8bbb admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:08.643674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 887/1794] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:08 2022] GET /identity/v3/roles => generated 1427 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:08.648173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34de271-fa69-4a26-ba44-b6cd9d586b56 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:08.668168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34de271-fa69-4a26-ba44-b6cd9d586b56 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:08.686541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa6ff5-5521-4817-a9cf-18289d325ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:08.693477 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa6ff5-5521-4817-a9cf-18289d325ea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:08.695112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa6ff5-5521-4817-a9cf-18289d325ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:08.697388 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa6ff5-5521-4817-a9cf-18289d325ea3 admin admin] PATH_INFO: `/v3/projects/5dcf994c0f9c4575b4f24faf935e4f45/users/dc6a52d1e93e4951aa2b48e7510173d4/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:08.707274 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa6ff5-5521-4817-a9cf-18289d325ea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dcf994c0f9c4575b4f24faf935e4f45, user_id=dc6a52d1e93e4951aa2b48e7510173d4, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:08.730820 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa6ff5-5521-4817-a9cf-18289d325ea3 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:08.785816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 908/1795] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:33:08 2022] PUT /identity/v3/projects/5dcf994c0f9c4575b4f24faf935e4f45/users/dc6a52d1e93e4951aa2b48e7510173d4/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:33:08.801693 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230aefc-cae8-475b-97e6-4f3bb9d5d6e3 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:08.802070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230aefc-cae8-475b-97e6-4f3bb9d5d6e3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:08.802426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230aefc-cae8-475b-97e6-4f3bb9d5d6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:08.876912 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8230aefc-cae8-475b-97e6-4f3bb9d5d6e3 None None] MFA Rules not processed for user `dc6a52d1e93e4951aa2b48e7510173d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:08.882660 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8230aefc-cae8-475b-97e6-4f3bb9d5d6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:09.118604 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 888/1796] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:09.132551 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa6ff5-5521-4817-a9cf-18289d325ea3 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:09.211108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa6ff5-5521-4817-a9cf-18289d325ea3 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:09.215334 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702b5bd-44a4-4825-869b-443f772df83e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:09.216139 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702b5bd-44a4-4825-869b-443f772df83e service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:09.216488 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702b5bd-44a4-4825-869b-443f772df83e service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:09.216769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702b5bd-44a4-4825-869b-443f772df83e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:09.240182 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f702b5bd-44a4-4825-869b-443f772df83e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:09.241756 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f702b5bd-44a4-4825-869b-443f772df83e service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:09.320874 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 909/1797] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:09 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:09.440433 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230aefc-cae8-475b-97e6-4f3bb9d5d6e3 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:09.457272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230aefc-cae8-475b-97e6-4f3bb9d5d6e3 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:09.462850 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58291e4d-2675-4d05-acf9-d88a9a4fd4dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:09.464007 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58291e4d-2675-4d05-acf9-d88a9a4fd4dd service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:09.464657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58291e4d-2675-4d05-acf9-d88a9a4fd4dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:09.465458 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58291e4d-2675-4d05-acf9-d88a9a4fd4dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:09.659231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58291e4d-2675-4d05-acf9-d88a9a4fd4dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:09.661106 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58291e4d-2675-4d05-acf9-d88a9a4fd4dd service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:09.690783 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 889/1798] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:10.768235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702b5bd-44a4-4825-869b-443f772df83e service cinder] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:10.769963 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-f702b5bd-44a4-4825-869b-443f772df83e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:11.014097 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d164de-2f30-4c7b-ae4f-37ca289d3440 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:11.015104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d164de-2f30-4c7b-ae4f-37ca289d3440 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:11.015588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d164de-2f30-4c7b-ae4f-37ca289d3440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:11.055230 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:11.115884 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702b5bd-44a4-4825-869b-443f772df83e service cinder] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:11.140737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0ac92-699e-4163-82bc-a30d8c203ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:11.141936 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0ac92-699e-4163-82bc-a30d8c203ad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:11.142429 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0ac92-699e-4163-82bc-a30d8c203ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:11.142939 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0ac92-699e-4163-82bc-a30d8c203ad6 admin admin] PATH_INFO: `/v3/users/0a08d4bc8dde46748c82dd3a5409f230` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:11.146294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c0ac92-699e-4163-82bc-a30d8c203ad6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a08d4bc8dde46748c82dd3a5409f230)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:11.148684 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c0ac92-699e-4163-82bc-a30d8c203ad6 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:11.162542 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-59d164de-2f30-4c7b-ae4f-37ca289d3440 None None] MFA Rules not processed for user `efe6e59228bd49bfbff25a11eb682426`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:11.164192 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:11.167769 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-59d164de-2f30-4c7b-ae4f-37ca289d3440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:11.270306 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-66c0ac92-699e-4163-82bc-a30d8c203ad6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a08d4bc8dde46748c82dd3a5409f230'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:11.271352 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-66c0ac92-699e-4163-82bc-a30d8c203ad6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a08d4bc8dde46748c82dd3a5409f230'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:11.297553 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-66c0ac92-699e-4163-82bc-a30d8c203ad6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a08d4bc8dde46748c82dd3a5409f230'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:11.318756 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-66c0ac92-699e-4163-82bc-a30d8c203ad6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a08d4bc8dde46748c82dd3a5409f230'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:11.334992 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-66c0ac92-699e-4163-82bc-a30d8c203ad6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a08d4bc8dde46748c82dd3a5409f230'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:11.342749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 910/1799] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:33:10 2022] DELETE /identity/v3/users/0a08d4bc8dde46748c82dd3a5409f230 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:11.347364 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624f53a-cf8f-4e27-8835-f2d917bb195a None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:11.347792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624f53a-cf8f-4e27-8835-f2d917bb195a None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:11.348201 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624f53a-cf8f-4e27-8835-f2d917bb195a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:11.361822 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 890/1800] 10.222.0.121 () {58 vars in 1014 bytes} [Thu Aug 4 10:33:11 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:11.408793 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-d624f53a-cf8f-4e27-8835-f2d917bb195a None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:11.417988 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d624f53a-cf8f-4e27-8835-f2d917bb195a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:11.708593 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d164de-2f30-4c7b-ae4f-37ca289d3440 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:11.718185 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-59d164de-2f30-4c7b-ae4f-37ca289d3440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:11.721810 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 911/1801] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:11 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:33:11.826956 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624f53a-cf8f-4e27-8835-f2d917bb195a None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:11.832046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d624f53a-cf8f-4e27-8835-f2d917bb195a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:11.950337 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d164de-2f30-4c7b-ae4f-37ca289d3440 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:12.004541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624f53a-cf8f-4e27-8835-f2d917bb195a None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:12.015840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb3c8b-bbe7-4720-a372-7ff930bb8f16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:12.017424 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb3c8b-bbe7-4720-a372-7ff930bb8f16 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:12.017927 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb3c8b-bbe7-4720-a372-7ff930bb8f16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:12.018430 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb3c8b-bbe7-4720-a372-7ff930bb8f16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:12.036642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb3c8b-bbe7-4720-a372-7ff930bb8f16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:12.042046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb3c8b-bbe7-4720-a372-7ff930bb8f16 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:12.056533 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 912/1802] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:11 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:12.140771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8793196f-10ec-4243-9f6e-96a938781d2b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e16e26784601425c9883acd91f65becc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:12.143549 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793196f-10ec-4243-9f6e-96a938781d2b service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:12.144160 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793196f-10ec-4243-9f6e-96a938781d2b service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:12.144592 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793196f-10ec-4243-9f6e-96a938781d2b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:12.149152 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8793196f-10ec-4243-9f6e-96a938781d2b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:12.156880 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:12.281934 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8793196f-10ec-4243-9f6e-96a938781d2b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:12.283747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8793196f-10ec-4243-9f6e-96a938781d2b service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:12.309547 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 891/1803] 10.222.0.121 () {60 vars in 1389 bytes} [Thu Aug 4 10:33:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 10:33:13.113617 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb3c8b-bbe7-4720-a372-7ff930bb8f16 service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:13.115885 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-97eb3c8b-bbe7-4720-a372-7ff930bb8f16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:13.152907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb3c8b-bbe7-4720-a372-7ff930bb8f16 service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:13.182038 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beee699-cd6a-4068-8a02-b79484fc588d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:13.185771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beee699-cd6a-4068-8a02-b79484fc588d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:13.186274 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beee699-cd6a-4068-8a02-b79484fc588d admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:13.186712 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beee699-cd6a-4068-8a02-b79484fc588d admin admin] PATH_INFO: `/v3/projects/9f9794c4379f4aa5a930a8dd20be7628` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:13.196284 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beee699-cd6a-4068-8a02-b79484fc588d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f9794c4379f4aa5a930a8dd20be7628)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:13.209448 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beee699-cd6a-4068-8a02-b79484fc588d admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:13.277519 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:13.315690 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-9beee699-cd6a-4068-8a02-b79484fc588d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f9794c4379f4aa5a930a8dd20be7628', 'request_id': 'req-9beee699-cd6a-4068-8a02-b79484fc588d'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:13.329576 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-9beee699-cd6a-4068-8a02-b79484fc588d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f9794c4379f4aa5a930a8dd20be7628', 'request_id': 'req-9beee699-cd6a-4068-8a02-b79484fc588d'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:13.338806 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-9beee699-cd6a-4068-8a02-b79484fc588d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f9794c4379f4aa5a930a8dd20be7628', 'request_id': 'req-9beee699-cd6a-4068-8a02-b79484fc588d'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:13.386413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-9beee699-cd6a-4068-8a02-b79484fc588d admin admin] The token cache is being invalidate because project 9f9794c4379f4aa5a930a8dd20be7628 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:33:13.386992 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-9beee699-cd6a-4068-8a02-b79484fc588d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:13.387896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-9beee699-cd6a-4068-8a02-b79484fc588d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:13.390698 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 913/1804] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:33:13 2022] DELETE /identity/v3/projects/9f9794c4379f4aa5a930a8dd20be7628 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:13.398344 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6fd54-643c-4a3e-8b64-649445f1a150 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:13.398818 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6fd54-643c-4a3e-8b64-649445f1a150 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:13.399178 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6fd54-643c-4a3e-8b64-649445f1a150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:13.467989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d3b6fd54-643c-4a3e-8b64-649445f1a150 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:13.477133 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d3b6fd54-643c-4a3e-8b64-649445f1a150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:13.678962 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 892/1805] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:13.688741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dc978-0d13-4533-9057-e12051a8c762 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:13.689309 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dc978-0d13-4533-9057-e12051a8c762 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:13.689746 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dc978-0d13-4533-9057-e12051a8c762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:13.730774 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-2c6dc978-0d13-4533-9057-e12051a8c762 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:13.732847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-2c6dc978-0d13-4533-9057-e12051a8c762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:13.743506 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 914/1806] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:13.748226 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6fd54-643c-4a3e-8b64-649445f1a150 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:13.761137 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6fd54-643c-4a3e-8b64-649445f1a150 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:13.797595 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91100e80-1865-4c08-b444-201c1547a535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:13.798907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91100e80-1865-4c08-b444-201c1547a535 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:13.799450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91100e80-1865-4c08-b444-201c1547a535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:13.799760 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91100e80-1865-4c08-b444-201c1547a535 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:13.802041 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91100e80-1865-4c08-b444-201c1547a535 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:33:13.804566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91100e80-1865-4c08-b444-201c1547a535 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:13.812266 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91100e80-1865-4c08-b444-201c1547a535 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:13.861013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 893/1807] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:33:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:33:13.862540 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6dc978-0d13-4533-9057-e12051a8c762 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:13.878842 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6dc978-0d13-4533-9057-e12051a8c762 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:13.896749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb1254-9b53-4041-8cd1-b872667923d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:13.897923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb1254-9b53-4041-8cd1-b872667923d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:13.898529 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb1254-9b53-4041-8cd1-b872667923d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:13.900128 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb1254-9b53-4041-8cd1-b872667923d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:13.901970 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb1254-9b53-4041-8cd1-b872667923d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:13.906885 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb1254-9b53-4041-8cd1-b872667923d9 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:13.999521 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 915/1808] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:33:13 2022] POST /identity/v3/projects => generated 380 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:14.003468 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91100e80-1865-4c08-b444-201c1547a535 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:14.015929 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91100e80-1865-4c08-b444-201c1547a535 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:14.128892 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5887f49-d15c-4f47-b1c7-00392f1c2e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:14.133706 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5887f49-d15c-4f47-b1c7-00392f1c2e6d admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:14.134528 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5887f49-d15c-4f47-b1c7-00392f1c2e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:14.135078 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5887f49-d15c-4f47-b1c7-00392f1c2e6d admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:14.136483 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5887f49-d15c-4f47-b1c7-00392f1c2e6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:14.139168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5887f49-d15c-4f47-b1c7-00392f1c2e6d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:14.176529 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 894/1809] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:33:13 2022] POST /identity/v3/users => generated 291 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:14.179624 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb1254-9b53-4041-8cd1-b872667923d9 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:14.209124 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb1254-9b53-4041-8cd1-b872667923d9 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:14.237242 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935bd22-be7f-4e5d-b20c-aeabbe569e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:14.238196 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935bd22-be7f-4e5d-b20c-aeabbe569e14 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:14.238487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935bd22-be7f-4e5d-b20c-aeabbe569e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:14.238764 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935bd22-be7f-4e5d-b20c-aeabbe569e14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:14.239615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935bd22-be7f-4e5d-b20c-aeabbe569e14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:14.241314 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935bd22-be7f-4e5d-b20c-aeabbe569e14 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:14.258817 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 916/1810] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:14 2022] GET /identity/v3/roles => generated 1427 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:14.260089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5887f49-d15c-4f47-b1c7-00392f1c2e6d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:14.272131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5887f49-d15c-4f47-b1c7-00392f1c2e6d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:14.278210 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26677deb-8b09-4fa8-bf93-74fa92409578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:14.278944 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26677deb-8b09-4fa8-bf93-74fa92409578 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:14.279299 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26677deb-8b09-4fa8-bf93-74fa92409578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:14.279579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26677deb-8b09-4fa8-bf93-74fa92409578 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:14.280347 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26677deb-8b09-4fa8-bf93-74fa92409578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:14.282035 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26677deb-8b09-4fa8-bf93-74fa92409578 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:14.294255 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 895/1811] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:14 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:14.298698 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935bd22-be7f-4e5d-b20c-aeabbe569e14 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:14.310160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935bd22-be7f-4e5d-b20c-aeabbe569e14 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:14.316088 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a3746d-889a-4f11-89df-cb8342692580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:14.316800 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3746d-889a-4f11-89df-cb8342692580 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:14.317140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3746d-889a-4f11-89df-cb8342692580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:14.318379 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3746d-889a-4f11-89df-cb8342692580 admin admin] PATH_INFO: `/v3/projects/790b407f5a7c499588fa5570be3ea6d0/users/1bc4111d250a4576949c5a53446c5a21/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:14.338588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a3746d-889a-4f11-89df-cb8342692580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=790b407f5a7c499588fa5570be3ea6d0, user_id=1bc4111d250a4576949c5a53446c5a21, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:14.341766 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a3746d-889a-4f11-89df-cb8342692580 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:14.364173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 917/1812] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:33:14 2022] PUT /identity/v3/projects/790b407f5a7c499588fa5570be3ea6d0/users/1bc4111d250a4576949c5a53446c5a21/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:14.375010 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3c619-d469-4183-a520-16a1a9781ae0 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:14.375559 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3c619-d469-4183-a520-16a1a9781ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:14.375995 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3c619-d469-4183-a520-16a1a9781ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:14.399487 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:14.408999 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-07e3c619-d469-4183-a520-16a1a9781ae0 None None] MFA Rules not processed for user `1bc4111d250a4576949c5a53446c5a21`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:14.410713 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-07e3c619-d469-4183-a520-16a1a9781ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:14.484856 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 896/1813] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:14.489407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a3746d-889a-4f11-89df-cb8342692580 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:14.575863 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a3746d-889a-4f11-89df-cb8342692580 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:14.587498 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f01c49-f020-4c28-a8bb-4295ca1e3a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:14.588618 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f01c49-f020-4c28-a8bb-4295ca1e3a3c admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:14.588960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f01c49-f020-4c28-a8bb-4295ca1e3a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:14.589317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f01c49-f020-4c28-a8bb-4295ca1e3a3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:14.590128 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f01c49-f020-4c28-a8bb-4295ca1e3a3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:14.592189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f01c49-f020-4c28-a8bb-4295ca1e3a3c admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:14.624900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 918/1814] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:33:14 2022] POST /identity/v3/projects => generated 380 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:14.629299 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3c619-d469-4183-a520-16a1a9781ae0 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:14.642928 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3c619-d469-4183-a520-16a1a9781ae0 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:14.714332 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425a7d1-cc14-4963-8efc-710697a5eef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:14.716696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425a7d1-cc14-4963-8efc-710697a5eef7 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:14.717250 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425a7d1-cc14-4963-8efc-710697a5eef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:14.718399 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425a7d1-cc14-4963-8efc-710697a5eef7 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:14.718399 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425a7d1-cc14-4963-8efc-710697a5eef7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:14.720867 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425a7d1-cc14-4963-8efc-710697a5eef7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:14.736524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f01c49-f020-4c28-a8bb-4295ca1e3a3c admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:14.738427 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-f8f01c49-f020-4c28-a8bb-4295ca1e3a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:14.746146 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 897/1815] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:33:14 2022] POST /identity/v3/users => generated 297 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:14.750119 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425a7d1-cc14-4963-8efc-710697a5eef7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:14.752354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f01c49-f020-4c28-a8bb-4295ca1e3a3c admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:14.757906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08f033-a027-447d-86cc-a13d14328757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:14.758596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08f033-a027-447d-86cc-a13d14328757 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:14.759063 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08f033-a027-447d-86cc-a13d14328757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:14.759353 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08f033-a027-447d-86cc-a13d14328757 admin admin] PATH_INFO: `/v3/users/15d4ade3a53540388df740f14dd67631` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:14.760359 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425a7d1-cc14-4963-8efc-710697a5eef7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:14.760702 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a08f033-a027-447d-86cc-a13d14328757 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15d4ade3a53540388df740f14dd67631)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:14.762717 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a08f033-a027-447d-86cc-a13d14328757 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:14.765240 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649cc94-0486-4498-9d81-597b59a5a3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:14.765896 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649cc94-0486-4498-9d81-597b59a5a3b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:14.766210 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649cc94-0486-4498-9d81-597b59a5a3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:14.766485 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649cc94-0486-4498-9d81-597b59a5a3b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:14.767188 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649cc94-0486-4498-9d81-597b59a5a3b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:14.768816 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649cc94-0486-4498-9d81-597b59a5a3b0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:14.783313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 898/1816] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:14 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:33:14.785594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649cc94-0486-4498-9d81-597b59a5a3b0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:14.797509 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:14.798203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649cc94-0486-4498-9d81-597b59a5a3b0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:14.803345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc752763-2a8d-449a-bcc5-0d96582e4c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:14.804063 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc752763-2a8d-449a-bcc5-0d96582e4c45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:14.804344 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc752763-2a8d-449a-bcc5-0d96582e4c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:14.804627 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc752763-2a8d-449a-bcc5-0d96582e4c45 admin admin] PATH_INFO: `/v3/projects/a0f229a997a54468b18a3e865f53fb18/users/0940a23ade614a928bd84dda95be01c0/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:14.818804 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-4a08f033-a027-447d-86cc-a13d14328757 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15d4ade3a53540388df740f14dd67631'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:14.819638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-4a08f033-a027-447d-86cc-a13d14328757 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15d4ade3a53540388df740f14dd67631'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:14.821215 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc752763-2a8d-449a-bcc5-0d96582e4c45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0f229a997a54468b18a3e865f53fb18, user_id=0940a23ade614a928bd84dda95be01c0, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:14.823995 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc752763-2a8d-449a-bcc5-0d96582e4c45 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:14.834642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-4a08f033-a027-447d-86cc-a13d14328757 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15d4ade3a53540388df740f14dd67631'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:14.839667 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 899/1817] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:33:14 2022] PUT /identity/v3/projects/a0f229a997a54468b18a3e865f53fb18/users/0940a23ade614a928bd84dda95be01c0/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:14.842008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-4a08f033-a027-447d-86cc-a13d14328757 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15d4ade3a53540388df740f14dd67631'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:14.843837 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc752763-2a8d-449a-bcc5-0d96582e4c45 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:14.852601 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-4a08f033-a027-447d-86cc-a13d14328757 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15d4ade3a53540388df740f14dd67631'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:14.861587 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 919/1818] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:33:14 2022] DELETE /identity/v3/users/15d4ade3a53540388df740f14dd67631 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:33:14.866165 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e66dd-3ff5-496d-8be8-9f2a1e699512 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:14.866453 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e66dd-3ff5-496d-8be8-9f2a1e699512 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:14.866732 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e66dd-3ff5-496d-8be8-9f2a1e699512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:14.889147 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-c72e66dd-3ff5-496d-8be8-9f2a1e699512 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:14.890678 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c72e66dd-3ff5-496d-8be8-9f2a1e699512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:14.915866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:14.922366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc752763-2a8d-449a-bcc5-0d96582e4c45 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:14.965041 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 920/1819] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:14.977169 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e66dd-3ff5-496d-8be8-9f2a1e699512 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:14.979455 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c72e66dd-3ff5-496d-8be8-9f2a1e699512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:15.025184 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d1ba6-1ce0-456e-8bd4-8fa4b7182f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:15.025835 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d1ba6-1ce0-456e-8bd4-8fa4b7182f31 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:15.026121 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d1ba6-1ce0-456e-8bd4-8fa4b7182f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:15.026396 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d1ba6-1ce0-456e-8bd4-8fa4b7182f31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:15.027114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6d1ba6-1ce0-456e-8bd4-8fa4b7182f31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:15.028908 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6d1ba6-1ce0-456e-8bd4-8fa4b7182f31 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:15.065090 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 900/1820] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:14 2022] GET /identity/v3/roles => generated 1427 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:15.069915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d1ba6-1ce0-456e-8bd4-8fa4b7182f31 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:15.076118 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ec6d1ba6-1ce0-456e-8bd4-8fa4b7182f31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:15.091219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d1ba6-1ce0-456e-8bd4-8fa4b7182f31 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:15.091938 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e66dd-3ff5-496d-8be8-9f2a1e699512 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:15.097458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38378600-7353-42a3-b415-0d21127f8542 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:15.098119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38378600-7353-42a3-b415-0d21127f8542 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:15.098404 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38378600-7353-42a3-b415-0d21127f8542 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:15.098683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38378600-7353-42a3-b415-0d21127f8542 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:15.119355 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8efac9a-4c06-4c83-a669-76751c3c3c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:15.125593 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8efac9a-4c06-4c83-a669-76751c3c3c39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:15.126687 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8efac9a-4c06-4c83-a669-76751c3c3c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:15.127143 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8efac9a-4c06-4c83-a669-76751c3c3c39 admin admin] PATH_INFO: `/v3/projects/a0f229a997a54468b18a3e865f53fb18/users/0940a23ade614a928bd84dda95be01c0/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:15.136544 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8efac9a-4c06-4c83-a669-76751c3c3c39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0f229a997a54468b18a3e865f53fb18, user_id=0940a23ade614a928bd84dda95be01c0, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:15.140000 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8efac9a-4c06-4c83-a669-76751c3c3c39 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:15.145987 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38378600-7353-42a3-b415-0d21127f8542 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:15.147988 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38378600-7353-42a3-b415-0d21127f8542 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:15.165483 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 901/1821] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:33:15 2022] PUT /identity/v3/projects/a0f229a997a54468b18a3e865f53fb18/users/0940a23ade614a928bd84dda95be01c0/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:15.180319 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a465ab-658f-49b1-be4e-429e922cc0cc None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:15.180718 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a465ab-658f-49b1-be4e-429e922cc0cc None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:15.186300 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a465ab-658f-49b1-be4e-429e922cc0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:15.186462 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 921/1822] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:14 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:33:15.246783 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-45a465ab-658f-49b1-be4e-429e922cc0cc None None] MFA Rules not processed for user `0940a23ade614a928bd84dda95be01c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:15.249096 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-45a465ab-658f-49b1-be4e-429e922cc0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:15.418561 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 902/1823] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:15 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:15.431321 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38378600-7353-42a3-b415-0d21127f8542 service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:15.432739 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-38378600-7353-42a3-b415-0d21127f8542 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:15.601555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:15.621662 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38378600-7353-42a3-b415-0d21127f8542 service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:15.780421 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f06682-612b-4714-a12d-c0be191ff27c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:15.781403 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f06682-612b-4714-a12d-c0be191ff27c service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:15.781917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f06682-612b-4714-a12d-c0be191ff27c service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:15.782351 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f06682-612b-4714-a12d-c0be191ff27c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:15.814675 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f06682-612b-4714-a12d-c0be191ff27c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:15.821124 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f06682-612b-4714-a12d-c0be191ff27c service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:15.834907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 922/1824] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:15 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:15.969479 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a465ab-658f-49b1-be4e-429e922cc0cc None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:15.986959 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a465ab-658f-49b1-be4e-429e922cc0cc None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:15.991718 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bee5f5-49aa-4278-a98e-225f2bc3121e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:15.993185 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bee5f5-49aa-4278-a98e-225f2bc3121e service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:15.993678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bee5f5-49aa-4278-a98e-225f2bc3121e service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:15.994145 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bee5f5-49aa-4278-a98e-225f2bc3121e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:15.997198 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b4bee5f5-49aa-4278-a98e-225f2bc3121e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:16.140682 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f06682-612b-4714-a12d-c0be191ff27c service cinder] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:16.142304 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-23f06682-612b-4714-a12d-c0be191ff27c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:16.143418 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bee5f5-49aa-4278-a98e-225f2bc3121e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:16.154842 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bee5f5-49aa-4278-a98e-225f2bc3121e service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:16.173755 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 903/1825] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:15 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:33:16.325585 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f06682-612b-4714-a12d-c0be191ff27c service cinder] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:16.330804 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117b072-942a-44cf-97ff-6e0816fbf296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:16.331715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117b072-942a-44cf-97ff-6e0816fbf296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:16.332035 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117b072-942a-44cf-97ff-6e0816fbf296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:16.332578 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117b072-942a-44cf-97ff-6e0816fbf296 admin admin] PATH_INFO: `/v3/projects/7514c7229ff0406aa761c50cefd3492c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:16.333841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c117b072-942a-44cf-97ff-6e0816fbf296 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7514c7229ff0406aa761c50cefd3492c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:16.336889 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c117b072-942a-44cf-97ff-6e0816fbf296 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:16.481261 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c117b072-942a-44cf-97ff-6e0816fbf296 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7514c7229ff0406aa761c50cefd3492c', 'request_id': 'req-c117b072-942a-44cf-97ff-6e0816fbf296'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:16.504598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c117b072-942a-44cf-97ff-6e0816fbf296 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7514c7229ff0406aa761c50cefd3492c', 'request_id': 'req-c117b072-942a-44cf-97ff-6e0816fbf296'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:16.516709 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c117b072-942a-44cf-97ff-6e0816fbf296 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7514c7229ff0406aa761c50cefd3492c', 'request_id': 'req-c117b072-942a-44cf-97ff-6e0816fbf296'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:16.535333 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bee5f5-49aa-4278-a98e-225f2bc3121e service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:16.542308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c117b072-942a-44cf-97ff-6e0816fbf296 admin admin] The token cache is being invalidate because project 7514c7229ff0406aa761c50cefd3492c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:33:16.542603 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c117b072-942a-44cf-97ff-6e0816fbf296 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:16.543617 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c117b072-942a-44cf-97ff-6e0816fbf296 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:16.547930 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bee5f5-49aa-4278-a98e-225f2bc3121e service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:16.548144 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 923/1826] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:33:16 2022] DELETE /identity/v3/projects/7514c7229ff0406aa761c50cefd3492c => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:16.552812 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117b072-942a-44cf-97ff-6e0816fbf296 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:16.564430 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c117b072-942a-44cf-97ff-6e0816fbf296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:16.578126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:16.682869 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a719338-5296-4e63-92ea-bf2287702941 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:16.684327 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a719338-5296-4e63-92ea-bf2287702941 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:16.684627 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a719338-5296-4e63-92ea-bf2287702941 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:16.685013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a719338-5296-4e63-92ea-bf2287702941 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:16.688109 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0a719338-5296-4e63-92ea-bf2287702941 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:16.742171 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:16.805184 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117b072-942a-44cf-97ff-6e0816fbf296 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:16.817035 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd467527-b0bb-47d3-8167-b00631db92c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:16.817731 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd467527-b0bb-47d3-8167-b00631db92c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:16.818018 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd467527-b0bb-47d3-8167-b00631db92c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:16.818699 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd467527-b0bb-47d3-8167-b00631db92c2 admin admin] PATH_INFO: `/v3/users/dc6a52d1e93e4951aa2b48e7510173d4` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:16.821473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd467527-b0bb-47d3-8167-b00631db92c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc6a52d1e93e4951aa2b48e7510173d4)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:16.822809 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd467527-b0bb-47d3-8167-b00631db92c2 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:17.016551 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a719338-5296-4e63-92ea-bf2287702941 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:17.016551 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a719338-5296-4e63-92ea-bf2287702941 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:17.036480 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-fd467527-b0bb-47d3-8167-b00631db92c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc6a52d1e93e4951aa2b48e7510173d4'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:17.038581 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-fd467527-b0bb-47d3-8167-b00631db92c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc6a52d1e93e4951aa2b48e7510173d4'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:17.091992 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 904/1827] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:16 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:17.098383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abf763-6a2e-4cce-b709-1406ed03a433 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:17.099091 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abf763-6a2e-4cce-b709-1406ed03a433 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:17.102600 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abf763-6a2e-4cce-b709-1406ed03a433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:17.129541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-fd467527-b0bb-47d3-8167-b00631db92c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc6a52d1e93e4951aa2b48e7510173d4'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:17.137196 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-71abf763-6a2e-4cce-b709-1406ed03a433 None None] MFA Rules not processed for user `90c7eaf4500f4ef496ba72405522006d`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:17.138681 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-71abf763-6a2e-4cce-b709-1406ed03a433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:17.199436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-fd467527-b0bb-47d3-8167-b00631db92c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc6a52d1e93e4951aa2b48e7510173d4'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:17.210994 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 905/1828] 10.222.0.107 () {58 vars in 1020 bytes} [Thu Aug 4 10:33:17 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:17.257488 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-fd467527-b0bb-47d3-8167-b00631db92c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc6a52d1e93e4951aa2b48e7510173d4'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:17.285790 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 924/1829] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:33:16 2022] DELETE /identity/v3/users/dc6a52d1e93e4951aa2b48e7510173d4 => generated 0 bytes in 733 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:18.349407 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abf763-6a2e-4cce-b709-1406ed03a433 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:18.370792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abf763-6a2e-4cce-b709-1406ed03a433 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:18.482136 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fcdb25-6b80-4554-9cb5-dcc64814c7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:18.482136 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcdb25-6b80-4554-9cb5-dcc64814c7ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:18.482136 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcdb25-6b80-4554-9cb5-dcc64814c7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:18.482136 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcdb25-6b80-4554-9cb5-dcc64814c7ab admin admin] PATH_INFO: `/v3/projects/5dcf994c0f9c4575b4f24faf935e4f45` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:18.483984 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fcdb25-6b80-4554-9cb5-dcc64814c7ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dcf994c0f9c4575b4f24faf935e4f45)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:18.488057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fcdb25-6b80-4554-9cb5-dcc64814c7ab admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:18.528305 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:18.646378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a4fcdb25-6b80-4554-9cb5-dcc64814c7ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dcf994c0f9c4575b4f24faf935e4f45', 'request_id': 'req-a4fcdb25-6b80-4554-9cb5-dcc64814c7ab'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:18.666556 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a4fcdb25-6b80-4554-9cb5-dcc64814c7ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dcf994c0f9c4575b4f24faf935e4f45', 'request_id': 'req-a4fcdb25-6b80-4554-9cb5-dcc64814c7ab'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:18.684714 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a4fcdb25-6b80-4554-9cb5-dcc64814c7ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dcf994c0f9c4575b4f24faf935e4f45', 'request_id': 'req-a4fcdb25-6b80-4554-9cb5-dcc64814c7ab'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:18.724012 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a4fcdb25-6b80-4554-9cb5-dcc64814c7ab admin admin] The token cache is being invalidate because project 5dcf994c0f9c4575b4f24faf935e4f45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:33:18.724293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a4fcdb25-6b80-4554-9cb5-dcc64814c7ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:18.725082 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a4fcdb25-6b80-4554-9cb5-dcc64814c7ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:18.727070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 906/1830] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:33:18 2022] DELETE /identity/v3/projects/5dcf994c0f9c4575b4f24faf935e4f45 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:18.735639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70832350-de21-4aa5-8eaf-c5680f9ce67e None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:18.736258 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70832350-de21-4aa5-8eaf-c5680f9ce67e None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:18.740031 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70832350-de21-4aa5-8eaf-c5680f9ce67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:18.783736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-70832350-de21-4aa5-8eaf-c5680f9ce67e None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:18.783736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-70832350-de21-4aa5-8eaf-c5680f9ce67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:18.861090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 925/1831] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:18 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:18.870155 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b712ad-6112-4a40-9fb2-2525ea4a7e3c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:18.870478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b712ad-6112-4a40-9fb2-2525ea4a7e3c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:18.870783 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b712ad-6112-4a40-9fb2-2525ea4a7e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:18.915037 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-52b712ad-6112-4a40-9fb2-2525ea4a7e3c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:18.915037 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-52b712ad-6112-4a40-9fb2-2525ea4a7e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:18.927117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 907/1832] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:18 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:33:18.933237 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70832350-de21-4aa5-8eaf-c5680f9ce67e None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:18.946935 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70832350-de21-4aa5-8eaf-c5680f9ce67e None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:18.952478 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba7fd7-f061-4180-988d-c56efa82fbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:18.953137 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba7fd7-f061-4180-988d-c56efa82fbde admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:18.953425 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba7fd7-f061-4180-988d-c56efa82fbde admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:18.953788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba7fd7-f061-4180-988d-c56efa82fbde admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:18.954552 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ba7fd7-f061-4180-988d-c56efa82fbde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:33:18.954866 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ba7fd7-f061-4180-988d-c56efa82fbde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:18.956573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ba7fd7-f061-4180-988d-c56efa82fbde admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:18.975424 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 926/1833] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:33:18 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:33:18.979451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b712ad-6112-4a40-9fb2-2525ea4a7e3c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:18.990011 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b712ad-6112-4a40-9fb2-2525ea4a7e3c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:18.995788 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2f1bf-2be9-47d9-ba12-76e8cb1c0ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:18.996494 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2f1bf-2be9-47d9-ba12-76e8cb1c0ce0 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:18.996784 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2f1bf-2be9-47d9-ba12-76e8cb1c0ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:18.997063 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2f1bf-2be9-47d9-ba12-76e8cb1c0ce0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:18.997770 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2f1bf-2be9-47d9-ba12-76e8cb1c0ce0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:18.999779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2f1bf-2be9-47d9-ba12-76e8cb1c0ce0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:19.041217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 908/1834] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:33:18 2022] POST /identity/v3/projects => generated 402 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:19.042351 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba7fd7-f061-4180-988d-c56efa82fbde admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:19.054072 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba7fd7-f061-4180-988d-c56efa82fbde admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:19.111796 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:19.146170 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8acda-5b49-4437-b3d6-30777c465727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:19.146874 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8acda-5b49-4437-b3d6-30777c465727 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:19.147969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8acda-5b49-4437-b3d6-30777c465727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:19.148401 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8acda-5b49-4437-b3d6-30777c465727 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:19.149554 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a8acda-5b49-4437-b3d6-30777c465727 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:19.157637 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a8acda-5b49-4437-b3d6-30777c465727 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:19.191978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 927/1835] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:33:19 2022] POST /identity/v3/users => generated 302 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:19.192305 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2f1bf-2be9-47d9-ba12-76e8cb1c0ce0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:19.202924 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2f1bf-2be9-47d9-ba12-76e8cb1c0ce0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:19.208337 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad42af-0328-48fc-83f1-9965dded2496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:19.209066 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad42af-0328-48fc-83f1-9965dded2496 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:19.209660 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad42af-0328-48fc-83f1-9965dded2496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:19.210018 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad42af-0328-48fc-83f1-9965dded2496 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:19.210840 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad42af-0328-48fc-83f1-9965dded2496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:19.212702 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad42af-0328-48fc-83f1-9965dded2496 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:19.229214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8acda-5b49-4437-b3d6-30777c465727 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:19.229824 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 909/1836] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:19 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:33:19.251788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8acda-5b49-4437-b3d6-30777c465727 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:19.269212 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d7f95-3237-4276-b062-fa97dc5cc6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:19.270477 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d7f95-3237-4276-b062-fa97dc5cc6ba admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:19.270828 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d7f95-3237-4276-b062-fa97dc5cc6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:19.271204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d7f95-3237-4276-b062-fa97dc5cc6ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:19.272495 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d7f95-3237-4276-b062-fa97dc5cc6ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:19.275610 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d7f95-3237-4276-b062-fa97dc5cc6ba admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:19.290396 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 928/1837] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:19 2022] GET /identity/v3/roles => generated 1427 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:19.294866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad42af-0328-48fc-83f1-9965dded2496 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:19.307430 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad42af-0328-48fc-83f1-9965dded2496 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:19.317612 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bd67d-c0e5-4a11-9e57-4ef686a03d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:19.318222 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bd67d-c0e5-4a11-9e57-4ef686a03d51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:19.318511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bd67d-c0e5-4a11-9e57-4ef686a03d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:19.318762 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bd67d-c0e5-4a11-9e57-4ef686a03d51 admin admin] PATH_INFO: `/v3/projects/dac1a0260fd847c2820f2710773c1c57/users/5810f5759e964ac699becbff2eb27b64/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:19.343363 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bd67d-c0e5-4a11-9e57-4ef686a03d51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dac1a0260fd847c2820f2710773c1c57, user_id=5810f5759e964ac699becbff2eb27b64, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:19.346703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bd67d-c0e5-4a11-9e57-4ef686a03d51 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:19.371823 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 910/1838] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:33:19 2022] PUT /identity/v3/projects/dac1a0260fd847c2820f2710773c1c57/users/5810f5759e964ac699becbff2eb27b64/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:19.375835 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9756252-a113-4950-b558-5e2dee695a82 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:19.376192 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9756252-a113-4950-b558-5e2dee695a82 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:19.376438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9756252-a113-4950-b558-5e2dee695a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:19.414316 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-a9756252-a113-4950-b558-5e2dee695a82 None None] MFA Rules not processed for user `5810f5759e964ac699becbff2eb27b64`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:19.416069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-a9756252-a113-4950-b558-5e2dee695a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:19.504178 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 929/1839] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:19 2022] POST /identity/v3/auth/tokens => generated 3288 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:19.507337 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bd67d-c0e5-4a11-9e57-4ef686a03d51 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:19.622767 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bd67d-c0e5-4a11-9e57-4ef686a03d51 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:19.628052 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9936d7-7c34-45fa-a04a-07bcecedfdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:19.628694 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9936d7-7c34-45fa-a04a-07bcecedfdef admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:19.628980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9936d7-7c34-45fa-a04a-07bcecedfdef admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:19.629258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9936d7-7c34-45fa-a04a-07bcecedfdef admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:19.629951 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9936d7-7c34-45fa-a04a-07bcecedfdef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:19.631905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9936d7-7c34-45fa-a04a-07bcecedfdef admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:19.665050 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 911/1840] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:33:19 2022] POST /identity/v3/projects => generated 400 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:19.669352 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9756252-a113-4950-b558-5e2dee695a82 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:19.681103 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9756252-a113-4950-b558-5e2dee695a82 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:19.743980 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7631311-98ad-4196-8314-5cbaf3231e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:19.744741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7631311-98ad-4196-8314-5cbaf3231e9a admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:19.744910 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7631311-98ad-4196-8314-5cbaf3231e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:19.745217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7631311-98ad-4196-8314-5cbaf3231e9a admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:19.745919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7631311-98ad-4196-8314-5cbaf3231e9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:19.747865 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7631311-98ad-4196-8314-5cbaf3231e9a admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:19.773049 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 930/1841] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:33:19 2022] POST /identity/v3/users => generated 307 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:19.777137 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9936d7-7c34-45fa-a04a-07bcecedfdef admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:19.792310 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9936d7-7c34-45fa-a04a-07bcecedfdef admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:19.798877 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253437d-36a7-4514-9e5f-d2ee80dfd663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:19.799602 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253437d-36a7-4514-9e5f-d2ee80dfd663 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:19.800073 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253437d-36a7-4514-9e5f-d2ee80dfd663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:19.800687 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253437d-36a7-4514-9e5f-d2ee80dfd663 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:19.801263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b253437d-36a7-4514-9e5f-d2ee80dfd663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:19.803358 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b253437d-36a7-4514-9e5f-d2ee80dfd663 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:19.815832 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 912/1842] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:19 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:19.820611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7631311-98ad-4196-8314-5cbaf3231e9a admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:19.834715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7631311-98ad-4196-8314-5cbaf3231e9a admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:19.840615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd7eed2-82f6-4858-839c-d47c2c9b9564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:19.841322 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7eed2-82f6-4858-839c-d47c2c9b9564 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:19.841606 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7eed2-82f6-4858-839c-d47c2c9b9564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:19.841882 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7eed2-82f6-4858-839c-d47c2c9b9564 admin admin] PATH_INFO: `/v3/projects/2d68be97c74f4c6eaa31ba225e12ca45/users/4628ab5e3351465d8be2af0ce9d7c9d6/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:19.860556 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd7eed2-82f6-4858-839c-d47c2c9b9564 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d68be97c74f4c6eaa31ba225e12ca45, user_id=4628ab5e3351465d8be2af0ce9d7c9d6, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:19.863489 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd7eed2-82f6-4858-839c-d47c2c9b9564 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:19.889848 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 931/1843] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:33:19 2022] PUT /identity/v3/projects/2d68be97c74f4c6eaa31ba225e12ca45/users/4628ab5e3351465d8be2af0ce9d7c9d6/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:19.894304 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253437d-36a7-4514-9e5f-d2ee80dfd663 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:19.908448 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253437d-36a7-4514-9e5f-d2ee80dfd663 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:19.914387 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:19.986498 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6483c4e-35bb-4554-9156-64e00fcd6818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:19.987227 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6483c4e-35bb-4554-9156-64e00fcd6818 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:19.988224 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6483c4e-35bb-4554-9156-64e00fcd6818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:19.988953 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6483c4e-35bb-4554-9156-64e00fcd6818 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:19.991340 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6483c4e-35bb-4554-9156-64e00fcd6818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:19.992932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6483c4e-35bb-4554-9156-64e00fcd6818 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:20.004659 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 913/1844] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:19 2022] GET /identity/v3/roles => generated 1427 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:33:20.009125 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd7eed2-82f6-4858-839c-d47c2c9b9564 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:20.021228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd7eed2-82f6-4858-839c-d47c2c9b9564 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:20.026714 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36053e6d-54f8-4009-99a0-8d995a9176af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:20.027424 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36053e6d-54f8-4009-99a0-8d995a9176af admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:20.027715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36053e6d-54f8-4009-99a0-8d995a9176af admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:20.028032 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36053e6d-54f8-4009-99a0-8d995a9176af admin admin] PATH_INFO: `/v3/projects/2d68be97c74f4c6eaa31ba225e12ca45/users/4628ab5e3351465d8be2af0ce9d7c9d6/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:20.030397 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36053e6d-54f8-4009-99a0-8d995a9176af admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d68be97c74f4c6eaa31ba225e12ca45, user_id=4628ab5e3351465d8be2af0ce9d7c9d6, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:20.033350 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36053e6d-54f8-4009-99a0-8d995a9176af admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:20.044799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 932/1845] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:33:20 2022] PUT /identity/v3/projects/2d68be97c74f4c6eaa31ba225e12ca45/users/4628ab5e3351465d8be2af0ce9d7c9d6/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:20.056744 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0e412-118f-4609-8e27-0fa5f60ff51f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:20.057041 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0e412-118f-4609-8e27-0fa5f60ff51f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:20.057327 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0e412-118f-4609-8e27-0fa5f60ff51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:20.091119 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c4f0e412-118f-4609-8e27-0fa5f60ff51f None None] MFA Rules not processed for user `4628ab5e3351465d8be2af0ce9d7c9d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:20.092443 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c4f0e412-118f-4609-8e27-0fa5f60ff51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:20.180992 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 914/1846] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:20 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:20.192116 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36053e6d-54f8-4009-99a0-8d995a9176af admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:20.193930 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-36053e6d-54f8-4009-99a0-8d995a9176af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:20.200850 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:20.260424 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36053e6d-54f8-4009-99a0-8d995a9176af admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:20.270783 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21435ea0-b3c6-4171-a2be-ba82f885eedc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:20.271603 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21435ea0-b3c6-4171-a2be-ba82f885eedc service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:20.271729 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21435ea0-b3c6-4171-a2be-ba82f885eedc service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:20.272167 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21435ea0-b3c6-4171-a2be-ba82f885eedc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:20.282874 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21435ea0-b3c6-4171-a2be-ba82f885eedc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:20.284383 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21435ea0-b3c6-4171-a2be-ba82f885eedc service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:20.296915 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 933/1847] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:20 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:20.383584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0e412-118f-4609-8e27-0fa5f60ff51f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:20.397746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0e412-118f-4609-8e27-0fa5f60ff51f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:20.401445 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf94cb-1b01-4933-b5db-fecc519cf400 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:20.402081 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf94cb-1b01-4933-b5db-fecc519cf400 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:20.402365 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf94cb-1b01-4933-b5db-fecc519cf400 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:20.402645 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf94cb-1b01-4933-b5db-fecc519cf400 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:20.465445 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf94cb-1b01-4933-b5db-fecc519cf400 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:20.466924 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf94cb-1b01-4933-b5db-fecc519cf400 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:20.477397 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 915/1848] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:20 2022] GET /identity/v3/auth/tokens => generated 3288 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:27.809236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf94cb-1b01-4933-b5db-fecc519cf400 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:27.810739 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21435ea0-b3c6-4171-a2be-ba82f885eedc service cinder] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:27.812915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2edf94cb-1b01-4933-b5db-fecc519cf400 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:27.815121 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-21435ea0-b3c6-4171-a2be-ba82f885eedc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:27.870482 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:27.922652 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf94cb-1b01-4933-b5db-fecc519cf400 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:27.930959 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043589d0-9bdd-4679-9901-f450bb7ee7b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:27.932427 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043589d0-9bdd-4679-9901-f450bb7ee7b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:27.933044 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043589d0-9bdd-4679-9901-f450bb7ee7b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:27.933451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043589d0-9bdd-4679-9901-f450bb7ee7b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:27.938071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-043589d0-9bdd-4679-9901-f450bb7ee7b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:27.940683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21435ea0-b3c6-4171-a2be-ba82f885eedc service cinder] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:27.945528 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc51fa-cd62-4f13-b077-3c89627c9a5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:27.946688 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc51fa-cd62-4f13-b077-3c89627c9a5b service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:27.947118 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc51fa-cd62-4f13-b077-3c89627c9a5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:27.947525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc51fa-cd62-4f13-b077-3c89627c9a5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:27.952145 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043589d0-9bdd-4679-9901-f450bb7ee7b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:27.953758 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043589d0-9bdd-4679-9901-f450bb7ee7b1 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:27.958733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc51fa-cd62-4f13-b077-3c89627c9a5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:27.960598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc51fa-cd62-4f13-b077-3c89627c9a5b service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:27.969542 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 916/1849] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:27 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:27.970845 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043589d0-9bdd-4679-9901-f450bb7ee7b1 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:27.973055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-043589d0-9bdd-4679-9901-f450bb7ee7b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:27.973215 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 934/1850] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:27 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:33:27.985413 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043589d0-9bdd-4679-9901-f450bb7ee7b1 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:27.989587 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ceb86b-3eeb-4ee8-a4f5-025e266df6cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:27.990307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ceb86b-3eeb-4ee8-a4f5-025e266df6cc service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:27.990540 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ceb86b-3eeb-4ee8-a4f5-025e266df6cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:27.990780 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ceb86b-3eeb-4ee8-a4f5-025e266df6cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:28.002167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ceb86b-3eeb-4ee8-a4f5-025e266df6cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:28.003637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ceb86b-3eeb-4ee8-a4f5-025e266df6cc service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:28.014271 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 917/1851] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:27 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:29.293184 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f4bf0-2efc-4e24-b500-95540224ee90 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:29.294267 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f4bf0-2efc-4e24-b500-95540224ee90 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:29.294549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f4bf0-2efc-4e24-b500-95540224ee90 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:29.296674 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 935/1852] 10.222.0.121 () {54 vars in 926 bytes} [Thu Aug 4 10:33:29 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:33:29.305267 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea86fb-94b0-449f-aa1e-360b1887f83d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:29.306274 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea86fb-94b0-449f-aa1e-360b1887f83d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:29.306653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea86fb-94b0-449f-aa1e-360b1887f83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:29.333146 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-afea86fb-94b0-449f-aa1e-360b1887f83d None None] MFA Rules not processed for user `0d947aeb749546309a8d4b7938087ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:29.334557 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-afea86fb-94b0-449f-aa1e-360b1887f83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:29.486016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 918/1853] 10.222.0.121 () {58 vars in 1023 bytes} [Thu Aug 4 10:33:29 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:29.498138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f4bf0-2efc-4e24-b500-95540224ee90 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:29.500270 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-031f4bf0-2efc-4e24-b500-95540224ee90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:29.531890 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f4bf0-2efc-4e24-b500-95540224ee90 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:29.540000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa915252-3a43-42c6-a60b-2e84278f07b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:29.541200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa915252-3a43-42c6-a60b-2e84278f07b2 service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:29.541659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa915252-3a43-42c6-a60b-2e84278f07b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:29.542154 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa915252-3a43-42c6-a60b-2e84278f07b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:29.555893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa915252-3a43-42c6-a60b-2e84278f07b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:29.559081 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa915252-3a43-42c6-a60b-2e84278f07b2 service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:29.577225 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 936/1854] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:29 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:30.245310 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4ed8e-15d3-488b-bf92-a49ba95349fc None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:30.245719 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4ed8e-15d3-488b-bf92-a49ba95349fc None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:30.245984 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4ed8e-15d3-488b-bf92-a49ba95349fc None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:30.247530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 919/1855] 10.222.0.121 () {54 vars in 926 bytes} [Thu Aug 4 10:33:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:33:30.264706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401df43-fa31-4c80-9917-285cf756f615 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:30.264934 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401df43-fa31-4c80-9917-285cf756f615 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:30.265236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401df43-fa31-4c80-9917-285cf756f615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:30.289122 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-b401df43-fa31-4c80-9917-285cf756f615 None None] MFA Rules not processed for user `0d947aeb749546309a8d4b7938087ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:30.290484 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-b401df43-fa31-4c80-9917-285cf756f615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:30.307449 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 937/1856] 10.222.0.121 () {58 vars in 1023 bytes} [Thu Aug 4 10:33:30 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:30.525883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe4ed8e-15d3-488b-bf92-a49ba95349fc None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:30.528522 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fbe4ed8e-15d3-488b-bf92-a49ba95349fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:30.559674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe4ed8e-15d3-488b-bf92-a49ba95349fc None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:30.567584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35bec5-0e7f-4f29-9abc-d0e4966906e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:30.568516 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35bec5-0e7f-4f29-9abc-d0e4966906e0 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:30.568982 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35bec5-0e7f-4f29-9abc-d0e4966906e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:30.569411 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35bec5-0e7f-4f29-9abc-d0e4966906e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:30.582899 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35bec5-0e7f-4f29-9abc-d0e4966906e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:30.584618 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35bec5-0e7f-4f29-9abc-d0e4966906e0 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:30.611271 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 920/1857] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:30 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:40.059229 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401df43-fa31-4c80-9917-285cf756f615 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:40.143486 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401df43-fa31-4c80-9917-285cf756f615 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:40.149230 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aeb2e2-d958-47de-b760-ff8fe5a85224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:40.149954 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aeb2e2-d958-47de-b760-ff8fe5a85224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:40.150258 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aeb2e2-d958-47de-b760-ff8fe5a85224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:40.150523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aeb2e2-d958-47de-b760-ff8fe5a85224 admin admin] PATH_INFO: `/v3/users/5810f5759e964ac699becbff2eb27b64` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:40.151777 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aeb2e2-d958-47de-b760-ff8fe5a85224 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5810f5759e964ac699becbff2eb27b64)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:40.153825 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aeb2e2-d958-47de-b760-ff8fe5a85224 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:40.200306 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:40.225724 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b7aeb2e2-d958-47de-b760-ff8fe5a85224 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5810f5759e964ac699becbff2eb27b64'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:40.226827 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b7aeb2e2-d958-47de-b760-ff8fe5a85224 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5810f5759e964ac699becbff2eb27b64'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:40.254785 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b7aeb2e2-d958-47de-b760-ff8fe5a85224 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5810f5759e964ac699becbff2eb27b64'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:40.263098 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b7aeb2e2-d958-47de-b760-ff8fe5a85224 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5810f5759e964ac699becbff2eb27b64'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:40.275598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b7aeb2e2-d958-47de-b760-ff8fe5a85224 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5810f5759e964ac699becbff2eb27b64'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:40.285171 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 938/1858] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:33:40 2022] DELETE /identity/v3/users/5810f5759e964ac699becbff2eb27b64 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:33:40.294530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f756e2-16e2-4063-b845-7ed1c8e29cdc None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:40.294530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f756e2-16e2-4063-b845-7ed1c8e29cdc None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:40.298153 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f756e2-16e2-4063-b845-7ed1c8e29cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:40.348553 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d3f756e2-16e2-4063-b845-7ed1c8e29cdc None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:40.350960 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d3f756e2-16e2-4063-b845-7ed1c8e29cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:40.492212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 921/1859] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:40 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:40.509361 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aeb2e2-d958-47de-b760-ff8fe5a85224 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:40.513299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-b7aeb2e2-d958-47de-b760-ff8fe5a85224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:40.533413 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f756e2-16e2-4063-b845-7ed1c8e29cdc None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:40.535685 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d3f756e2-16e2-4063-b845-7ed1c8e29cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:40.561035 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f756e2-16e2-4063-b845-7ed1c8e29cdc None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:40.573606 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4767e-8baa-4a3a-a850-cdd29191091e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:40.575167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4767e-8baa-4a3a-a850-cdd29191091e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:40.575677 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4767e-8baa-4a3a-a850-cdd29191091e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:40.576228 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4767e-8baa-4a3a-a850-cdd29191091e admin admin] PATH_INFO: `/v3/users/1bc4111d250a4576949c5a53446c5a21` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:40.578606 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4767e-8baa-4a3a-a850-cdd29191091e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bc4111d250a4576949c5a53446c5a21)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:40.582614 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4767e-8baa-4a3a-a850-cdd29191091e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:40.642524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aeb2e2-d958-47de-b760-ff8fe5a85224 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:40.648615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88b6fb-c0e1-48b4-87ea-a29c48454409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:40.649709 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88b6fb-c0e1-48b4-87ea-a29c48454409 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:40.650174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88b6fb-c0e1-48b4-87ea-a29c48454409 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:40.650843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88b6fb-c0e1-48b4-87ea-a29c48454409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:40.665982 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:40.666366 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c88b6fb-c0e1-48b4-87ea-a29c48454409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:40.667882 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c88b6fb-c0e1-48b4-87ea-a29c48454409 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:40.683552 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 939/1860] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:40 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:40.703285 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-30d4767e-8baa-4a3a-a850-cdd29191091e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bc4111d250a4576949c5a53446c5a21'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:40.717924 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-30d4767e-8baa-4a3a-a850-cdd29191091e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bc4111d250a4576949c5a53446c5a21'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:40.719252 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-30d4767e-8baa-4a3a-a850-cdd29191091e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bc4111d250a4576949c5a53446c5a21'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:40.759061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-30d4767e-8baa-4a3a-a850-cdd29191091e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bc4111d250a4576949c5a53446c5a21'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:40.761148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-30d4767e-8baa-4a3a-a850-cdd29191091e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bc4111d250a4576949c5a53446c5a21'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:40.810333 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 922/1861] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:33:40 2022] DELETE /identity/v3/users/1bc4111d250a4576949c5a53446c5a21 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:33:40.813233 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a372a08-2db3-4990-8214-be7183ffb898 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:40.814204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a372a08-2db3-4990-8214-be7183ffb898 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:40.814518 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a372a08-2db3-4990-8214-be7183ffb898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:40.864826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-3a372a08-2db3-4990-8214-be7183ffb898 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:40.867560 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-3a372a08-2db3-4990-8214-be7183ffb898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:41.030166 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 940/1862] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:40 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:41.044468 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4767e-8baa-4a3a-a850-cdd29191091e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:41.046375 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-30d4767e-8baa-4a3a-a850-cdd29191091e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:41.157693 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4767e-8baa-4a3a-a850-cdd29191091e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:41.164158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b4bcb-7dd2-4699-acf0-d83763735d8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:41.164945 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b4bcb-7dd2-4699-acf0-d83763735d8a service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:41.165236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b4bcb-7dd2-4699-acf0-d83763735d8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:41.165537 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b4bcb-7dd2-4699-acf0-d83763735d8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:41.182451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b4bcb-7dd2-4699-acf0-d83763735d8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:41.184741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b4bcb-7dd2-4699-acf0-d83763735d8a service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:41.200272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 923/1863] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:41 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:41.277574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a372a08-2db3-4990-8214-be7183ffb898 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:41.282814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-3a372a08-2db3-4990-8214-be7183ffb898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:41.310355 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a372a08-2db3-4990-8214-be7183ffb898 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:41.323202 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3cdee-7ff0-4131-ba79-643ee67c6107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:41.324947 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3cdee-7ff0-4131-ba79-643ee67c6107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:41.325210 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3cdee-7ff0-4131-ba79-643ee67c6107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:41.326120 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3cdee-7ff0-4131-ba79-643ee67c6107 admin admin] PATH_INFO: `/v3/projects/dac1a0260fd847c2820f2710773c1c57` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:41.328617 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3cdee-7ff0-4131-ba79-643ee67c6107 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dac1a0260fd847c2820f2710773c1c57)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:41.332796 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3cdee-7ff0-4131-ba79-643ee67c6107 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:41.415296 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:41.416883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0dd3cdee-7ff0-4131-ba79-643ee67c6107 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dac1a0260fd847c2820f2710773c1c57', 'request_id': 'req-0dd3cdee-7ff0-4131-ba79-643ee67c6107'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:41.432227 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0dd3cdee-7ff0-4131-ba79-643ee67c6107 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dac1a0260fd847c2820f2710773c1c57', 'request_id': 'req-0dd3cdee-7ff0-4131-ba79-643ee67c6107'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:41.452026 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0dd3cdee-7ff0-4131-ba79-643ee67c6107 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dac1a0260fd847c2820f2710773c1c57', 'request_id': 'req-0dd3cdee-7ff0-4131-ba79-643ee67c6107'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:41.472440 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0dd3cdee-7ff0-4131-ba79-643ee67c6107 admin admin] The token cache is being invalidate because project dac1a0260fd847c2820f2710773c1c57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:33:41.473012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0dd3cdee-7ff0-4131-ba79-643ee67c6107 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:41.477258 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0dd3cdee-7ff0-4131-ba79-643ee67c6107 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:41.486642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 941/1864] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:33:41 2022] DELETE /identity/v3/projects/dac1a0260fd847c2820f2710773c1c57 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:41.499262 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b4bcb-7dd2-4699-acf0-d83763735d8a service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:41.514411 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b4bcb-7dd2-4699-acf0-d83763735d8a service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:41.605218 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525b3f6-146b-460a-9f9c-77dc3398fd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:41.605898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525b3f6-146b-460a-9f9c-77dc3398fd8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:41.606187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525b3f6-146b-460a-9f9c-77dc3398fd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:41.606473 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525b3f6-146b-460a-9f9c-77dc3398fd8e admin admin] PATH_INFO: `/v3/users/4628ab5e3351465d8be2af0ce9d7c9d6` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:41.607784 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525b3f6-146b-460a-9f9c-77dc3398fd8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4628ab5e3351465d8be2af0ce9d7c9d6)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:41.609931 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525b3f6-146b-460a-9f9c-77dc3398fd8e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:41.682580 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8525b3f6-146b-460a-9f9c-77dc3398fd8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4628ab5e3351465d8be2af0ce9d7c9d6'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:41.690864 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:41.694452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8525b3f6-146b-460a-9f9c-77dc3398fd8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4628ab5e3351465d8be2af0ce9d7c9d6'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:41.696770 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8525b3f6-146b-460a-9f9c-77dc3398fd8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4628ab5e3351465d8be2af0ce9d7c9d6'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:41.716587 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8525b3f6-146b-460a-9f9c-77dc3398fd8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4628ab5e3351465d8be2af0ce9d7c9d6'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:41.717470 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8525b3f6-146b-460a-9f9c-77dc3398fd8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4628ab5e3351465d8be2af0ce9d7c9d6'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:41.742476 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 924/1865] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:33:41 2022] DELETE /identity/v3/users/4628ab5e3351465d8be2af0ce9d7c9d6 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:41.813991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3cdee-7ff0-4131-ba79-643ee67c6107 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:41.815800 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-0dd3cdee-7ff0-4131-ba79-643ee67c6107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:41.862586 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3cdee-7ff0-4131-ba79-643ee67c6107 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:41.958542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae4571-dbbb-4cd6-9e75-8080f4c555c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:41.959215 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae4571-dbbb-4cd6-9e75-8080f4c555c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:41.959532 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae4571-dbbb-4cd6-9e75-8080f4c555c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:41.959803 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae4571-dbbb-4cd6-9e75-8080f4c555c7 admin admin] PATH_INFO: `/v3/projects/790b407f5a7c499588fa5570be3ea6d0` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:41.961130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae4571-dbbb-4cd6-9e75-8080f4c555c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=790b407f5a7c499588fa5570be3ea6d0)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:41.964745 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae4571-dbbb-4cd6-9e75-8080f4c555c7 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:42.041179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c4ae4571-dbbb-4cd6-9e75-8080f4c555c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '790b407f5a7c499588fa5570be3ea6d0', 'request_id': 'req-c4ae4571-dbbb-4cd6-9e75-8080f4c555c7'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:42.053639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c4ae4571-dbbb-4cd6-9e75-8080f4c555c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '790b407f5a7c499588fa5570be3ea6d0', 'request_id': 'req-c4ae4571-dbbb-4cd6-9e75-8080f4c555c7'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:42.063504 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c4ae4571-dbbb-4cd6-9e75-8080f4c555c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '790b407f5a7c499588fa5570be3ea6d0', 'request_id': 'req-c4ae4571-dbbb-4cd6-9e75-8080f4c555c7'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:42.089269 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c4ae4571-dbbb-4cd6-9e75-8080f4c555c7 admin admin] The token cache is being invalidate because project 790b407f5a7c499588fa5570be3ea6d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:33:42.089998 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c4ae4571-dbbb-4cd6-9e75-8080f4c555c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:42.091129 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c4ae4571-dbbb-4cd6-9e75-8080f4c555c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:42.096315 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 942/1866] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:33:41 2022] DELETE /identity/v3/projects/790b407f5a7c499588fa5570be3ea6d0 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:42.103648 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525b3f6-146b-460a-9f9c-77dc3398fd8e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:42.107099 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8525b3f6-146b-460a-9f9c-77dc3398fd8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:42.164279 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525b3f6-146b-460a-9f9c-77dc3398fd8e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:42.192806 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae4571-dbbb-4cd6-9e75-8080f4c555c7 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:42.194375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c4ae4571-dbbb-4cd6-9e75-8080f4c555c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:42.266867 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2209e1a-0841-473b-85a0-9eb1655237d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:42.267791 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2209e1a-0841-473b-85a0-9eb1655237d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:42.268193 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2209e1a-0841-473b-85a0-9eb1655237d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:42.268892 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2209e1a-0841-473b-85a0-9eb1655237d1 admin admin] PATH_INFO: `/v3/users/0940a23ade614a928bd84dda95be01c0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:42.269831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2209e1a-0841-473b-85a0-9eb1655237d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0940a23ade614a928bd84dda95be01c0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:42.272016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2209e1a-0841-473b-85a0-9eb1655237d1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:42.299275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae4571-dbbb-4cd6-9e75-8080f4c555c7 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:42.305600 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dba542-ab70-4850-8cb0-ecfaed4e8b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:42.306289 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dba542-ab70-4850-8cb0-ecfaed4e8b2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:42.306577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dba542-ab70-4850-8cb0-ecfaed4e8b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:42.306855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dba542-ab70-4850-8cb0-ecfaed4e8b2d admin admin] PATH_INFO: `/v3/projects/2d68be97c74f4c6eaa31ba225e12ca45` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:42.308332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dba542-ab70-4850-8cb0-ecfaed4e8b2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d68be97c74f4c6eaa31ba225e12ca45)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:42.310541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dba542-ab70-4850-8cb0-ecfaed4e8b2d admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:42.320806 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:42.344849 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b2209e1a-0841-473b-85a0-9eb1655237d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0940a23ade614a928bd84dda95be01c0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:42.358921 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b2209e1a-0841-473b-85a0-9eb1655237d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0940a23ade614a928bd84dda95be01c0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:42.360419 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b2209e1a-0841-473b-85a0-9eb1655237d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0940a23ade614a928bd84dda95be01c0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:42.370229 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-24dba542-ab70-4850-8cb0-ecfaed4e8b2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d68be97c74f4c6eaa31ba225e12ca45', 'request_id': 'req-24dba542-ab70-4850-8cb0-ecfaed4e8b2d'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:42.377426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b2209e1a-0841-473b-85a0-9eb1655237d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0940a23ade614a928bd84dda95be01c0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:42.378573 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b2209e1a-0841-473b-85a0-9eb1655237d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0940a23ade614a928bd84dda95be01c0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:42.378922 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-24dba542-ab70-4850-8cb0-ecfaed4e8b2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d68be97c74f4c6eaa31ba225e12ca45', 'request_id': 'req-24dba542-ab70-4850-8cb0-ecfaed4e8b2d'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:42.386086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-24dba542-ab70-4850-8cb0-ecfaed4e8b2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d68be97c74f4c6eaa31ba225e12ca45', 'request_id': 'req-24dba542-ab70-4850-8cb0-ecfaed4e8b2d'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:42.398593 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 925/1867] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:33:42 2022] DELETE /identity/v3/users/0940a23ade614a928bd84dda95be01c0 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:42.400674 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-24dba542-ab70-4850-8cb0-ecfaed4e8b2d admin admin] The token cache is being invalidate because project 2d68be97c74f4c6eaa31ba225e12ca45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:33:42.400968 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-24dba542-ab70-4850-8cb0-ecfaed4e8b2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:42.401639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-24dba542-ab70-4850-8cb0-ecfaed4e8b2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:42.403461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 943/1868] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:33:42 2022] DELETE /identity/v3/projects/2d68be97c74f4c6eaa31ba225e12ca45 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:33:42.409527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e2783-33f4-40d4-9f1e-9b94f042aab0 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:42.409999 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e2783-33f4-40d4-9f1e-9b94f042aab0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:42.410493 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e2783-33f4-40d4-9f1e-9b94f042aab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:42.439401 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e25e2783-33f4-40d4-9f1e-9b94f042aab0 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:42.440828 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e25e2783-33f4-40d4-9f1e-9b94f042aab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:42.505858 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:42.524221 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056eec8-4691-424b-8532-cead2d670b94 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:42.524550 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056eec8-4691-424b-8532-cead2d670b94 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:42.524858 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056eec8-4691-424b-8532-cead2d670b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:42.528922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 926/1869] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:42.547844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-f056eec8-4691-424b-8532-cead2d670b94 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:42.550151 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-f056eec8-4691-424b-8532-cead2d670b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:42.564759 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 944/1870] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:42.574162 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e2783-33f4-40d4-9f1e-9b94f042aab0 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:42.585230 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e2783-33f4-40d4-9f1e-9b94f042aab0 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:42.592886 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc227ad-f470-4b25-aa24-f1eb99263457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:42.593549 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc227ad-f470-4b25-aa24-f1eb99263457 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:42.593864 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc227ad-f470-4b25-aa24-f1eb99263457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:42.594179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc227ad-f470-4b25-aa24-f1eb99263457 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:42.594933 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc227ad-f470-4b25-aa24-f1eb99263457 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:33:42.595248 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc227ad-f470-4b25-aa24-f1eb99263457 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:42.596925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc227ad-f470-4b25-aa24-f1eb99263457 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:42.610389 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 927/1871] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:33:42 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:33:42.621992 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056eec8-4691-424b-8532-cead2d670b94 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:42.631514 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056eec8-4691-424b-8532-cead2d670b94 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:42.636249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd12d02-78f2-4bac-b70e-7d552994be69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:42.636880 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd12d02-78f2-4bac-b70e-7d552994be69 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:42.637158 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd12d02-78f2-4bac-b70e-7d552994be69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:42.637434 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd12d02-78f2-4bac-b70e-7d552994be69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:42.638108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd12d02-78f2-4bac-b70e-7d552994be69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:42.640060 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd12d02-78f2-4bac-b70e-7d552994be69 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:42.670478 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 945/1872] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:33:42 2022] POST /identity/v3/projects => generated 384 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:42.674992 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc227ad-f470-4b25-aa24-f1eb99263457 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:42.686171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc227ad-f470-4b25-aa24-f1eb99263457 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:42.740246 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd12d02-78f2-4bac-b70e-7d552994be69 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:42.742119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-bfd12d02-78f2-4bac-b70e-7d552994be69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:42.746530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad0483-69ed-4f30-96ab-c63d5caba6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:42.747241 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad0483-69ed-4f30-96ab-c63d5caba6b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:42.747604 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad0483-69ed-4f30-96ab-c63d5caba6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:42.748145 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad0483-69ed-4f30-96ab-c63d5caba6b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:42.748840 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad0483-69ed-4f30-96ab-c63d5caba6b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:42.750998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad0483-69ed-4f30-96ab-c63d5caba6b0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:42.755457 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd12d02-78f2-4bac-b70e-7d552994be69 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:42.761022 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070267f-c21a-4e43-a264-8f9f1b34e9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:42.762635 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070267f-c21a-4e43-a264-8f9f1b34e9a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:42.763027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070267f-c21a-4e43-a264-8f9f1b34e9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:42.763866 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070267f-c21a-4e43-a264-8f9f1b34e9a9 admin admin] PATH_INFO: `/v3/projects/a0f229a997a54468b18a3e865f53fb18` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:42.765342 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070267f-c21a-4e43-a264-8f9f1b34e9a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0f229a997a54468b18a3e865f53fb18)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:42.767575 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070267f-c21a-4e43-a264-8f9f1b34e9a9 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:42.779929 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 928/1873] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:33:42 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:42.784239 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad0483-69ed-4f30-96ab-c63d5caba6b0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:42.796118 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad0483-69ed-4f30-96ab-c63d5caba6b0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:42.802168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c6b4a-eca3-4e9f-a4ed-dac1e463fb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:42.803080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c6b4a-eca3-4e9f-a4ed-dac1e463fb4a admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:42.803537 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c6b4a-eca3-4e9f-a4ed-dac1e463fb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:42.803788 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c6b4a-eca3-4e9f-a4ed-dac1e463fb4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:42.804743 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c6b4a-eca3-4e9f-a4ed-dac1e463fb4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:42.806702 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c6b4a-eca3-4e9f-a4ed-dac1e463fb4a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:42.820000 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 929/1874] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:42 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:33:42.823457 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c6b4a-eca3-4e9f-a4ed-dac1e463fb4a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:42.830263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-4070267f-c21a-4e43-a264-8f9f1b34e9a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0f229a997a54468b18a3e865f53fb18', 'request_id': 'req-4070267f-c21a-4e43-a264-8f9f1b34e9a9'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:42.836445 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c6b4a-eca3-4e9f-a4ed-dac1e463fb4a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:42.840342 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-4070267f-c21a-4e43-a264-8f9f1b34e9a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0f229a997a54468b18a3e865f53fb18', 'request_id': 'req-4070267f-c21a-4e43-a264-8f9f1b34e9a9'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:42.846872 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-4070267f-c21a-4e43-a264-8f9f1b34e9a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0f229a997a54468b18a3e865f53fb18', 'request_id': 'req-4070267f-c21a-4e43-a264-8f9f1b34e9a9'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:42.859885 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-4070267f-c21a-4e43-a264-8f9f1b34e9a9 admin admin] The token cache is being invalidate because project a0f229a997a54468b18a3e865f53fb18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:33:42.860695 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-4070267f-c21a-4e43-a264-8f9f1b34e9a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:42.862292 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-4070267f-c21a-4e43-a264-8f9f1b34e9a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:42.867317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 946/1875] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:33:42 2022] DELETE /identity/v3/projects/a0f229a997a54468b18a3e865f53fb18 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:33:42.870060 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f23eba-2b7a-4139-9b8f-f0ab4a794863 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:42.871565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f23eba-2b7a-4139-9b8f-f0ab4a794863 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:42.871849 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f23eba-2b7a-4139-9b8f-f0ab4a794863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:42.907189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-a1f23eba-2b7a-4139-9b8f-f0ab4a794863 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:42.908813 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-a1f23eba-2b7a-4139-9b8f-f0ab4a794863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:42.911192 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043b8cd-b726-4b2b-98a2-a0df7775ea52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:42.911860 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043b8cd-b726-4b2b-98a2-a0df7775ea52 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:42.912241 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043b8cd-b726-4b2b-98a2-a0df7775ea52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:42.912531 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043b8cd-b726-4b2b-98a2-a0df7775ea52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:42.913260 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043b8cd-b726-4b2b-98a2-a0df7775ea52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:42.915239 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043b8cd-b726-4b2b-98a2-a0df7775ea52 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:42.918674 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 947/1876] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:42.928123 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb67aa-0019-4ec2-a094-23891bdd2b9d None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:42.928418 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb67aa-0019-4ec2-a094-23891bdd2b9d None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:42.928705 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb67aa-0019-4ec2-a094-23891bdd2b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:42.935637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 930/1877] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:42 2022] GET /identity/v3/roles => generated 1427 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:42.936815 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043b8cd-b726-4b2b-98a2-a0df7775ea52 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:42.939358 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f043b8cd-b726-4b2b-98a2-a0df7775ea52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:42.966580 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043b8cd-b726-4b2b-98a2-a0df7775ea52 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:42.966580 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a9db6-f2b2-429c-bd6a-bf5a68105f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:42.966580 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a9db6-f2b2-429c-bd6a-bf5a68105f5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:42.966580 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a9db6-f2b2-429c-bd6a-bf5a68105f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:42.967041 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a9db6-f2b2-429c-bd6a-bf5a68105f5a admin admin] PATH_INFO: `/v3/projects/7550e8e1e78546188e03306c30e14690/users/8e742800fc5d4fbeb7487390312f8176/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:42.967117 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-b3eb67aa-0019-4ec2-a094-23891bdd2b9d None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:42.967117 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-b3eb67aa-0019-4ec2-a094-23891bdd2b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:42.972458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 948/1878] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:33:42.977043 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb67aa-0019-4ec2-a094-23891bdd2b9d None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:42.978608 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a9db6-f2b2-429c-bd6a-bf5a68105f5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7550e8e1e78546188e03306c30e14690, user_id=8e742800fc5d4fbeb7487390312f8176, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:42.981381 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a9db6-f2b2-429c-bd6a-bf5a68105f5a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:42.986960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb67aa-0019-4ec2-a094-23891bdd2b9d None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:42.992588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4bcf50-db2d-472d-952d-a2e91e9a5ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:42.993235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bcf50-db2d-472d-952d-a2e91e9a5ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:42.993640 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bcf50-db2d-472d-952d-a2e91e9a5ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:42.993944 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bcf50-db2d-472d-952d-a2e91e9a5ef8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:42.994635 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4bcf50-db2d-472d-952d-a2e91e9a5ef8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:33:42.994943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4bcf50-db2d-472d-952d-a2e91e9a5ef8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:42.996461 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 931/1879] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:33:42 2022] PUT /identity/v3/projects/7550e8e1e78546188e03306c30e14690/users/8e742800fc5d4fbeb7487390312f8176/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:43.004372 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4bcf50-db2d-472d-952d-a2e91e9a5ef8 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:43.007442 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8d8b3-1a55-4926-ad9c-c09db06256da None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:43.008029 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8d8b3-1a55-4926-ad9c-c09db06256da None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:43.008270 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8d8b3-1a55-4926-ad9c-c09db06256da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:43.012402 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 949/1880] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:33:42 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:33:43.013414 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4bcf50-db2d-472d-952d-a2e91e9a5ef8 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:43.023376 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4bcf50-db2d-472d-952d-a2e91e9a5ef8 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:43.038567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a9b8d8b3-1a55-4926-ad9c-c09db06256da None None] MFA Rules not processed for user `8e742800fc5d4fbeb7487390312f8176`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:43.039826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a9b8d8b3-1a55-4926-ad9c-c09db06256da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:43.043912 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:43.059854 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:43.086043 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e35bf7-f16c-498a-bd8a-e5bb4129408c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:43.086584 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e35bf7-f16c-498a-bd8a-e5bb4129408c admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:43.086869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e35bf7-f16c-498a-bd8a-e5bb4129408c admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:43.087156 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e35bf7-f16c-498a-bd8a-e5bb4129408c admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:43.087842 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e35bf7-f16c-498a-bd8a-e5bb4129408c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:43.089903 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e35bf7-f16c-498a-bd8a-e5bb4129408c admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:43.120052 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 932/1881] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:33:43.121945 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8d8b3-1a55-4926-ad9c-c09db06256da None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:43.127421 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 950/1882] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:33:43 2022] POST /identity/v3/projects => generated 374 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:43.131447 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e35bf7-f16c-498a-bd8a-e5bb4129408c admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:43.134378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8d8b3-1a55-4926-ad9c-c09db06256da None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:43.201812 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763902f-98b5-4c51-9dd7-61b43efb9740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:43.203119 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763902f-98b5-4c51-9dd7-61b43efb9740 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:43.203119 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763902f-98b5-4c51-9dd7-61b43efb9740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:43.203119 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763902f-98b5-4c51-9dd7-61b43efb9740 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:43.203801 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0763902f-98b5-4c51-9dd7-61b43efb9740 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:43.205778 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0763902f-98b5-4c51-9dd7-61b43efb9740 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:43.210850 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e35bf7-f16c-498a-bd8a-e5bb4129408c admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:43.216951 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf3577b-995a-4c4a-9ef5-bace2456505e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:43.218344 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3577b-995a-4c4a-9ef5-bace2456505e admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:43.218700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3577b-995a-4c4a-9ef5-bace2456505e admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:43.219286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3577b-995a-4c4a-9ef5-bace2456505e admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:43.220531 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf3577b-995a-4c4a-9ef5-bace2456505e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:43.222917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf3577b-995a-4c4a-9ef5-bace2456505e admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:43.248048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 933/1883] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:33:43 2022] POST /identity/v3/projects => generated 384 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:33:43.251558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 951/1884] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:33:43 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:33:43.254620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763902f-98b5-4c51-9dd7-61b43efb9740 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:43.255024 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf3577b-995a-4c4a-9ef5-bace2456505e admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:43.271987 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf3577b-995a-4c4a-9ef5-bace2456505e admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:43.361164 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8412f92-0af5-48bc-be0f-6768846f3e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:43.361878 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8412f92-0af5-48bc-be0f-6768846f3e13 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:43.362101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8412f92-0af5-48bc-be0f-6768846f3e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:43.362384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8412f92-0af5-48bc-be0f-6768846f3e13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:43.363121 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8412f92-0af5-48bc-be0f-6768846f3e13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:43.363853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763902f-98b5-4c51-9dd7-61b43efb9740 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:43.365566 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8412f92-0af5-48bc-be0f-6768846f3e13 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:43.386773 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 952/1885] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:43 2022] GET /identity/v3/roles => generated 1427 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:43.389106 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8412f92-0af5-48bc-be0f-6768846f3e13 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:43.399661 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c216a47-47a5-4fb0-aa2f-71bebc769c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:43.400923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c216a47-47a5-4fb0-aa2f-71bebc769c6f admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:43.401530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c216a47-47a5-4fb0-aa2f-71bebc769c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:43.402146 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c216a47-47a5-4fb0-aa2f-71bebc769c6f admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:43.408938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c216a47-47a5-4fb0-aa2f-71bebc769c6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:43.416576 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c216a47-47a5-4fb0-aa2f-71bebc769c6f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:43.427265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8412f92-0af5-48bc-be0f-6768846f3e13 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:43.433812 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e19da4-57e8-45ed-99b3-a7aba0a4ecab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:43.435093 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e19da4-57e8-45ed-99b3-a7aba0a4ecab admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:43.435697 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e19da4-57e8-45ed-99b3-a7aba0a4ecab admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:43.436184 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e19da4-57e8-45ed-99b3-a7aba0a4ecab admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:43.437066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e19da4-57e8-45ed-99b3-a7aba0a4ecab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:43.439213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e19da4-57e8-45ed-99b3-a7aba0a4ecab admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:43.456754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 953/1886] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:43 2022] GET /identity/v3/roles => generated 1427 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:43.462803 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e19da4-57e8-45ed-99b3-a7aba0a4ecab admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:43.467704 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 934/1887] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:33:43 2022] POST /identity/v3/users => generated 299 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:43.472819 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c216a47-47a5-4fb0-aa2f-71bebc769c6f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:43.484291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e19da4-57e8-45ed-99b3-a7aba0a4ecab admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:43.489964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc083c7-1640-4cf8-82f4-23c126ed793c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:43.490312 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c216a47-47a5-4fb0-aa2f-71bebc769c6f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:43.490815 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc083c7-1640-4cf8-82f4-23c126ed793c admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:43.491182 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc083c7-1640-4cf8-82f4-23c126ed793c admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:43.491510 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc083c7-1640-4cf8-82f4-23c126ed793c admin admin] PATH_INFO: `/v3/projects/fca3936fbfa54eb08d1cc818dc6be125/users/32f87fb821524b4d9f3b9091faacc174/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:43.498457 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca960a9c-1b2d-4f41-b1ea-55144d065422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:43.499414 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca960a9c-1b2d-4f41-b1ea-55144d065422 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:43.499879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca960a9c-1b2d-4f41-b1ea-55144d065422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:43.500467 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca960a9c-1b2d-4f41-b1ea-55144d065422 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:43.501506 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca960a9c-1b2d-4f41-b1ea-55144d065422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:43.503842 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca960a9c-1b2d-4f41-b1ea-55144d065422 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:43.520276 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 935/1888] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:43 2022] GET /identity/v3/roles => generated 1427 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:33:43.524680 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc083c7-1640-4cf8-82f4-23c126ed793c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fca3936fbfa54eb08d1cc818dc6be125, user_id=32f87fb821524b4d9f3b9091faacc174, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:43.524902 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca960a9c-1b2d-4f41-b1ea-55144d065422 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:43.529878 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc083c7-1640-4cf8-82f4-23c126ed793c admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:43.536162 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca960a9c-1b2d-4f41-b1ea-55144d065422 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:43.541301 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a06a6-efe1-4d3d-a357-a240843bbe4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:43.542103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a06a6-efe1-4d3d-a357-a240843bbe4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:43.542514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a06a6-efe1-4d3d-a357-a240843bbe4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:43.542930 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a06a6-efe1-4d3d-a357-a240843bbe4b admin admin] PATH_INFO: `/v3/projects/006426cc90754abba6136e045277b67c/users/7a21d2b7c0cd4809a1892e1bbe681940/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:43.551997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:43.561267 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 954/1889] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:33:43 2022] PUT /identity/v3/projects/fca3936fbfa54eb08d1cc818dc6be125/users/32f87fb821524b4d9f3b9091faacc174/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:43.573199 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375b2f4-81c2-492e-959a-50963159f3da None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:43.573599 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375b2f4-81c2-492e-959a-50963159f3da None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:43.573829 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375b2f4-81c2-492e-959a-50963159f3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:43.577601 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a06a6-efe1-4d3d-a357-a240843bbe4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=006426cc90754abba6136e045277b67c, user_id=7a21d2b7c0cd4809a1892e1bbe681940, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:43.580189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a06a6-efe1-4d3d-a357-a240843bbe4b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:43.592648 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:43.599704 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 936/1890] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:33:43 2022] PUT /identity/v3/projects/006426cc90754abba6136e045277b67c/users/7a21d2b7c0cd4809a1892e1bbe681940/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:43.605225 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a06a6-efe1-4d3d-a357-a240843bbe4b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:43.637030 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-0375b2f4-81c2-492e-959a-50963159f3da None None] MFA Rules not processed for user `32f87fb821524b4d9f3b9091faacc174`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:43.638623 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-0375b2f4-81c2-492e-959a-50963159f3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:43.732980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a06a6-efe1-4d3d-a357-a240843bbe4b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:43.745612 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95589caf-820f-483e-821f-93c9f401ec06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:43.745809 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 955/1891] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:33:43.745918 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95589caf-820f-483e-821f-93c9f401ec06 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:43.745918 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95589caf-820f-483e-821f-93c9f401ec06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:43.745918 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95589caf-820f-483e-821f-93c9f401ec06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:43.745918 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95589caf-820f-483e-821f-93c9f401ec06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:43.745918 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95589caf-820f-483e-821f-93c9f401ec06 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:43.757056 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0375b2f4-81c2-492e-959a-50963159f3da None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:43.757596 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 937/1892] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:43 2022] GET /identity/v3/roles => generated 1427 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:43.758572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-0375b2f4-81c2-492e-959a-50963159f3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:43.765486 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95589caf-820f-483e-821f-93c9f401ec06 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:43.782708 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95589caf-820f-483e-821f-93c9f401ec06 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:43.788736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cccb0-1d80-44d5-81d8-4516d49f5840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:43.789473 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cccb0-1d80-44d5-81d8-4516d49f5840 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:43.789762 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cccb0-1d80-44d5-81d8-4516d49f5840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:43.790042 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cccb0-1d80-44d5-81d8-4516d49f5840 admin admin] PATH_INFO: `/v3/projects/006426cc90754abba6136e045277b67c/users/7a21d2b7c0cd4809a1892e1bbe681940/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:43.793046 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632cccb0-1d80-44d5-81d8-4516d49f5840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=006426cc90754abba6136e045277b67c, user_id=7a21d2b7c0cd4809a1892e1bbe681940, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:43.795748 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632cccb0-1d80-44d5-81d8-4516d49f5840 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:43.812196 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 938/1893] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:33:43 2022] PUT /identity/v3/projects/006426cc90754abba6136e045277b67c/users/7a21d2b7c0cd4809a1892e1bbe681940/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:33:43.825039 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5a59d-156f-42e2-af0b-40aaa34935e2 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:43.825886 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5a59d-156f-42e2-af0b-40aaa34935e2 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:43.826826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5a59d-156f-42e2-af0b-40aaa34935e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:43.831901 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0375b2f4-81c2-492e-959a-50963159f3da None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:43.861428 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ccd5a59d-156f-42e2-af0b-40aaa34935e2 None None] MFA Rules not processed for user `7a21d2b7c0cd4809a1892e1bbe681940`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:43.862704 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ccd5a59d-156f-42e2-af0b-40aaa34935e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:43.885132 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f89016-7356-44ce-9f3a-3237f3155b7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:43.886126 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f89016-7356-44ce-9f3a-3237f3155b7b service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:43.886497 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f89016-7356-44ce-9f3a-3237f3155b7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:43.886790 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f89016-7356-44ce-9f3a-3237f3155b7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:43.939036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 939/1894] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:43 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:43.950841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f89016-7356-44ce-9f3a-3237f3155b7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:43.951229 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5a59d-156f-42e2-af0b-40aaa34935e2 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:43.952544 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f89016-7356-44ce-9f3a-3237f3155b7b service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:43.952755 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ccd5a59d-156f-42e2-af0b-40aaa34935e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:43.957437 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:43.963560 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 956/1895] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:43.965002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5a59d-156f-42e2-af0b-40aaa34935e2 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:43.978478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb11fe-7d77-4578-8e1d-9892a047b030 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:43.979426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb11fe-7d77-4578-8e1d-9892a047b030 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:43.979733 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb11fe-7d77-4578-8e1d-9892a047b030 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:43.980163 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb11fe-7d77-4578-8e1d-9892a047b030 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:43.986109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f89016-7356-44ce-9f3a-3237f3155b7b service cinder] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:43.992764 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb11fe-7d77-4578-8e1d-9892a047b030 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:43.994762 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb11fe-7d77-4578-8e1d-9892a047b030 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:44.007644 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 940/1896] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:43 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:33:44.056577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f89016-7356-44ce-9f3a-3237f3155b7b service cinder] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:44.061525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c9e78-4116-4e8c-a111-279318dd4462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:44.062181 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c9e78-4116-4e8c-a111-279318dd4462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:44.062472 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c9e78-4116-4e8c-a111-279318dd4462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:44.062841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c9e78-4116-4e8c-a111-279318dd4462 admin admin] PATH_INFO: `/v3/users/32f87fb821524b4d9f3b9091faacc174` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:44.064172 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c9e78-4116-4e8c-a111-279318dd4462 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32f87fb821524b4d9f3b9091faacc174)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:44.066135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c9e78-4116-4e8c-a111-279318dd4462 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:44.112643 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb11fe-7d77-4578-8e1d-9892a047b030 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:44.122092 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb11fe-7d77-4578-8e1d-9892a047b030 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:44.125972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-085c9e78-4116-4e8c-a111-279318dd4462 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32f87fb821524b4d9f3b9091faacc174'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:44.127091 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb6966-0505-4786-a540-5d6f666fc599 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:44.127750 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb6966-0505-4786-a540-5d6f666fc599 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:44.127986 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-085c9e78-4116-4e8c-a111-279318dd4462 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32f87fb821524b4d9f3b9091faacc174'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:44.128179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb6966-0505-4786-a540-5d6f666fc599 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:44.128314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb6966-0505-4786-a540-5d6f666fc599 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:44.146843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-085c9e78-4116-4e8c-a111-279318dd4462 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32f87fb821524b4d9f3b9091faacc174'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:44.154825 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-085c9e78-4116-4e8c-a111-279318dd4462 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32f87fb821524b4d9f3b9091faacc174'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:44.164551 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-085c9e78-4116-4e8c-a111-279318dd4462 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32f87fb821524b4d9f3b9091faacc174'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:44.171732 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 957/1897] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:33:43 2022] DELETE /identity/v3/users/32f87fb821524b4d9f3b9091faacc174 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:33:44.176893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827eed1-f403-4b83-b8cd-84ccf95d7369 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:44.177188 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827eed1-f403-4b83-b8cd-84ccf95d7369 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:44.177475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827eed1-f403-4b83-b8cd-84ccf95d7369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:44.189611 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:44.197496 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-e827eed1-f403-4b83-b8cd-84ccf95d7369 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:44.198790 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-e827eed1-f403-4b83-b8cd-84ccf95d7369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:44.200993 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb6966-0505-4786-a540-5d6f666fc599 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:44.202468 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb6966-0505-4786-a540-5d6f666fc599 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:44.212956 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 941/1898] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:44 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:44.265600 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 958/1899] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:44 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:44.276593 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb6966-0505-4786-a540-5d6f666fc599 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:44.278144 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-53cb6966-0505-4786-a540-5d6f666fc599 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:44.359068 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb6966-0505-4786-a540-5d6f666fc599 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:44.414397 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9ecff-098e-4d8d-ac68-9a27f406cdb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:44.415046 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9ecff-098e-4d8d-ac68-9a27f406cdb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:44.415331 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9ecff-098e-4d8d-ac68-9a27f406cdb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:44.415636 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9ecff-098e-4d8d-ac68-9a27f406cdb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:44.426917 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9ecff-098e-4d8d-ac68-9a27f406cdb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:44.429013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9ecff-098e-4d8d-ac68-9a27f406cdb6 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:44.441709 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 942/1900] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:44 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:44.517151 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827eed1-f403-4b83-b8cd-84ccf95d7369 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:44.518707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-e827eed1-f403-4b83-b8cd-84ccf95d7369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:44.531826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827eed1-f403-4b83-b8cd-84ccf95d7369 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:44.538605 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9449b46-4414-43ca-acc2-d701c463cb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:44.539733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9449b46-4414-43ca-acc2-d701c463cb20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:44.539733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9449b46-4414-43ca-acc2-d701c463cb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:44.540093 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9449b46-4414-43ca-acc2-d701c463cb20 admin admin] PATH_INFO: `/v3/users/8e742800fc5d4fbeb7487390312f8176` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:44.541531 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9449b46-4414-43ca-acc2-d701c463cb20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e742800fc5d4fbeb7487390312f8176)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:44.543916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9449b46-4414-43ca-acc2-d701c463cb20 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:44.562339 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:44.603431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b9449b46-4414-43ca-acc2-d701c463cb20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e742800fc5d4fbeb7487390312f8176'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:44.604187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b9449b46-4414-43ca-acc2-d701c463cb20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e742800fc5d4fbeb7487390312f8176'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:44.617321 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b9449b46-4414-43ca-acc2-d701c463cb20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e742800fc5d4fbeb7487390312f8176'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:44.624440 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b9449b46-4414-43ca-acc2-d701c463cb20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e742800fc5d4fbeb7487390312f8176'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:44.633412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b9449b46-4414-43ca-acc2-d701c463cb20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e742800fc5d4fbeb7487390312f8176'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:44.639602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 959/1901] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:33:44 2022] DELETE /identity/v3/users/8e742800fc5d4fbeb7487390312f8176 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:44.644326 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f34425-e48e-436e-b5a7-6732af4f7bde None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:44.644937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f34425-e48e-436e-b5a7-6732af4f7bde None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:44.644937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f34425-e48e-436e-b5a7-6732af4f7bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:44.667371 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-78f34425-e48e-436e-b5a7-6732af4f7bde None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:44.668658 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-78f34425-e48e-436e-b5a7-6732af4f7bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:44.731786 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 943/1902] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:44 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:44.742357 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9449b46-4414-43ca-acc2-d701c463cb20 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:44.744107 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-b9449b46-4414-43ca-acc2-d701c463cb20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:44.820496 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9449b46-4414-43ca-acc2-d701c463cb20 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:44.827286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e9fc0-de13-4ff2-8012-2468783f608f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:44.828714 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e9fc0-de13-4ff2-8012-2468783f608f service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:44.829248 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e9fc0-de13-4ff2-8012-2468783f608f service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:44.829734 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e9fc0-de13-4ff2-8012-2468783f608f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:44.841860 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e9fc0-de13-4ff2-8012-2468783f608f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:44.843891 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e9fc0-de13-4ff2-8012-2468783f608f service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:44.858851 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 960/1903] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:44 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:44.958688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f34425-e48e-436e-b5a7-6732af4f7bde None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:44.961398 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-78f34425-e48e-436e-b5a7-6732af4f7bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:44.975794 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f34425-e48e-436e-b5a7-6732af4f7bde None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:44.982417 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5925e2-a8e5-40a0-8a3b-de6d742997f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:44.983157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5925e2-a8e5-40a0-8a3b-de6d742997f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:44.983472 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5925e2-a8e5-40a0-8a3b-de6d742997f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:44.983632 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5925e2-a8e5-40a0-8a3b-de6d742997f8 admin admin] PATH_INFO: `/v3/projects/fca3936fbfa54eb08d1cc818dc6be125` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:44.985159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5925e2-a8e5-40a0-8a3b-de6d742997f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fca3936fbfa54eb08d1cc818dc6be125)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:44.987678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5925e2-a8e5-40a0-8a3b-de6d742997f8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:45.049436 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:45.052934 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fc5925e2-a8e5-40a0-8a3b-de6d742997f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fca3936fbfa54eb08d1cc818dc6be125', 'request_id': 'req-fc5925e2-a8e5-40a0-8a3b-de6d742997f8'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:45.061779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fc5925e2-a8e5-40a0-8a3b-de6d742997f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fca3936fbfa54eb08d1cc818dc6be125', 'request_id': 'req-fc5925e2-a8e5-40a0-8a3b-de6d742997f8'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:45.068851 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fc5925e2-a8e5-40a0-8a3b-de6d742997f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fca3936fbfa54eb08d1cc818dc6be125', 'request_id': 'req-fc5925e2-a8e5-40a0-8a3b-de6d742997f8'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:45.086990 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fc5925e2-a8e5-40a0-8a3b-de6d742997f8 admin admin] The token cache is being invalidate because project fca3936fbfa54eb08d1cc818dc6be125 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:33:45.087307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fc5925e2-a8e5-40a0-8a3b-de6d742997f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:45.087984 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fc5925e2-a8e5-40a0-8a3b-de6d742997f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:45.089606 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 944/1904] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:33:44 2022] DELETE /identity/v3/projects/fca3936fbfa54eb08d1cc818dc6be125 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:45.095139 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a23e0-ec71-4fee-b84c-0382bc6ec0b0 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:45.095529 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a23e0-ec71-4fee-b84c-0382bc6ec0b0 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:45.095923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a23e0-ec71-4fee-b84c-0382bc6ec0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:45.117967 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-446a23e0-ec71-4fee-b84c-0382bc6ec0b0 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:45.119432 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-446a23e0-ec71-4fee-b84c-0382bc6ec0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:45.191254 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 961/1905] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:45 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:45.193954 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88458b7-eb67-482d-9dc6-75b5bb6e9e26 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:45.194254 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88458b7-eb67-482d-9dc6-75b5bb6e9e26 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:45.194543 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88458b7-eb67-482d-9dc6-75b5bb6e9e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:45.217431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a23e0-ec71-4fee-b84c-0382bc6ec0b0 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:45.219876 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-446a23e0-ec71-4fee-b84c-0382bc6ec0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:45.232774 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a88458b7-eb67-482d-9dc6-75b5bb6e9e26 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:45.234026 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a88458b7-eb67-482d-9dc6-75b5bb6e9e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:45.235670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a23e0-ec71-4fee-b84c-0382bc6ec0b0 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:45.244286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9689dc-3e37-4ddc-b339-7b64971abdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:45.245521 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9689dc-3e37-4ddc-b339-7b64971abdd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:45.245965 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 945/1906] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:45 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:45.248337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9689dc-3e37-4ddc-b339-7b64971abdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:45.250968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88458b7-eb67-482d-9dc6-75b5bb6e9e26 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:45.252608 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9689dc-3e37-4ddc-b339-7b64971abdd7 admin admin] PATH_INFO: `/v3/projects/7550e8e1e78546188e03306c30e14690` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:45.257365 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9689dc-3e37-4ddc-b339-7b64971abdd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7550e8e1e78546188e03306c30e14690)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:45.258437 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9689dc-3e37-4ddc-b339-7b64971abdd7 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:45.270086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88458b7-eb67-482d-9dc6-75b5bb6e9e26 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:45.277100 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbde16e-f70d-4047-8133-164e69d6b38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:45.278260 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbde16e-f70d-4047-8133-164e69d6b38c admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:45.278260 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbde16e-f70d-4047-8133-164e69d6b38c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:45.278450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbde16e-f70d-4047-8133-164e69d6b38c admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:45.279011 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbde16e-f70d-4047-8133-164e69d6b38c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:33:45.279514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbde16e-f70d-4047-8133-164e69d6b38c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:45.281508 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbde16e-f70d-4047-8133-164e69d6b38c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:45.296506 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 946/1907] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:33:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:33:45.303899 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbde16e-f70d-4047-8133-164e69d6b38c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:45.306416 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:45.317786 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbde16e-f70d-4047-8133-164e69d6b38c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:45.323322 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-ab9689dc-3e37-4ddc-b339-7b64971abdd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7550e8e1e78546188e03306c30e14690', 'request_id': 'req-ab9689dc-3e37-4ddc-b339-7b64971abdd7'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:45.334121 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-ab9689dc-3e37-4ddc-b339-7b64971abdd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7550e8e1e78546188e03306c30e14690', 'request_id': 'req-ab9689dc-3e37-4ddc-b339-7b64971abdd7'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:45.361394 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-ab9689dc-3e37-4ddc-b339-7b64971abdd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7550e8e1e78546188e03306c30e14690', 'request_id': 'req-ab9689dc-3e37-4ddc-b339-7b64971abdd7'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:45.376942 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-ab9689dc-3e37-4ddc-b339-7b64971abdd7 admin admin] The token cache is being invalidate because project 7550e8e1e78546188e03306c30e14690 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:33:45.377240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-ab9689dc-3e37-4ddc-b339-7b64971abdd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:45.377931 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-ab9689dc-3e37-4ddc-b339-7b64971abdd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:45.379520 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 962/1908] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:33:45 2022] DELETE /identity/v3/projects/7550e8e1e78546188e03306c30e14690 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:45.384434 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9689dc-3e37-4ddc-b339-7b64971abdd7 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:45.386669 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-ab9689dc-3e37-4ddc-b339-7b64971abdd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:45.394792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a319dda-ade0-4276-8c9a-3bb0467d677b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:45.395440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a319dda-ade0-4276-8c9a-3bb0467d677b admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:45.395728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a319dda-ade0-4276-8c9a-3bb0467d677b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:45.396033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a319dda-ade0-4276-8c9a-3bb0467d677b admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:45.396796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a319dda-ade0-4276-8c9a-3bb0467d677b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:45.398735 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a319dda-ade0-4276-8c9a-3bb0467d677b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:45.430245 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 947/1909] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:33:45 2022] POST /identity/v3/projects => generated 376 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:45.435901 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a319dda-ade0-4276-8c9a-3bb0467d677b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:45.440316 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5a319dda-ade0-4276-8c9a-3bb0467d677b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:45.461287 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9689dc-3e37-4ddc-b339-7b64971abdd7 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:45.512200 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a319dda-ade0-4276-8c9a-3bb0467d677b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:45.517365 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ccd73-5853-48f5-bfc9-b8d0afc8a8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:45.518016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ccd73-5853-48f5-bfc9-b8d0afc8a8e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:45.518634 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ccd73-5853-48f5-bfc9-b8d0afc8a8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:45.518634 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ccd73-5853-48f5-bfc9-b8d0afc8a8e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:45.519335 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ccd73-5853-48f5-bfc9-b8d0afc8a8e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:45.521321 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ccd73-5853-48f5-bfc9-b8d0afc8a8e9 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:45.525249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f189103-2850-415e-869f-f83a0fac31e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:45.526004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f189103-2850-415e-869f-f83a0fac31e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:45.526296 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f189103-2850-415e-869f-f83a0fac31e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:45.526575 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f189103-2850-415e-869f-f83a0fac31e3 admin admin] PATH_INFO: `/v3/users/7a21d2b7c0cd4809a1892e1bbe681940` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:45.527873 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f189103-2850-415e-869f-f83a0fac31e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a21d2b7c0cd4809a1892e1bbe681940)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:45.529906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f189103-2850-415e-869f-f83a0fac31e3 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:45.546577 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 948/1910] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:33:45 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:45.555788 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ccd73-5853-48f5-bfc9-b8d0afc8a8e9 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:45.568051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ccd73-5853-48f5-bfc9-b8d0afc8a8e9 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:45.574243 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae34255-5d73-42db-873e-a5087b111404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:45.575101 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae34255-5d73-42db-873e-a5087b111404 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:45.575697 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae34255-5d73-42db-873e-a5087b111404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:45.576007 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae34255-5d73-42db-873e-a5087b111404 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:45.576737 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae34255-5d73-42db-873e-a5087b111404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:45.578546 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae34255-5d73-42db-873e-a5087b111404 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:45.601748 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 949/1911] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:45 2022] GET /identity/v3/roles => generated 1427 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:45.605308 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae34255-5d73-42db-873e-a5087b111404 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:45.607123 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6f189103-2850-415e-869f-f83a0fac31e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a21d2b7c0cd4809a1892e1bbe681940'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:45.607879 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6f189103-2850-415e-869f-f83a0fac31e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a21d2b7c0cd4809a1892e1bbe681940'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:45.617050 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae34255-5d73-42db-873e-a5087b111404 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:45.622716 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57e5cd-654d-48f3-ad2c-7b83e13850d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:45.623364 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57e5cd-654d-48f3-ad2c-7b83e13850d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:45.623645 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57e5cd-654d-48f3-ad2c-7b83e13850d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:45.623922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57e5cd-654d-48f3-ad2c-7b83e13850d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:45.632929 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6f189103-2850-415e-869f-f83a0fac31e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a21d2b7c0cd4809a1892e1bbe681940'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:45.633340 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57e5cd-654d-48f3-ad2c-7b83e13850d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:45.633340 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57e5cd-654d-48f3-ad2c-7b83e13850d1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:45.633544 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:45.636716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6f189103-2850-415e-869f-f83a0fac31e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a21d2b7c0cd4809a1892e1bbe681940'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:45.639013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 950/1912] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:45 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:33:45.643742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57e5cd-654d-48f3-ad2c-7b83e13850d1 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:45.648828 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6f189103-2850-415e-869f-f83a0fac31e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a21d2b7c0cd4809a1892e1bbe681940'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:45.657026 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57e5cd-654d-48f3-ad2c-7b83e13850d1 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:45.659846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 963/1913] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:33:45 2022] DELETE /identity/v3/users/7a21d2b7c0cd4809a1892e1bbe681940 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:33:45.664173 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:45.765013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64a785-a459-45d4-a9cd-aedcf2bed54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:45.765812 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64a785-a459-45d4-a9cd-aedcf2bed54b admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:45.766097 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64a785-a459-45d4-a9cd-aedcf2bed54b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:45.766376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64a785-a459-45d4-a9cd-aedcf2bed54b admin admin] PATH_INFO: `/v3/projects/df56de1ea1be4affa107cded85597809/users/9ea8d0d05b2a41429ccc77cc291a31e6/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:45.800220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64a785-a459-45d4-a9cd-aedcf2bed54b admin admin] RBAC: Authorizing `identity:create_grant(project_id=df56de1ea1be4affa107cded85597809, user_id=9ea8d0d05b2a41429ccc77cc291a31e6, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:45.800220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64a785-a459-45d4-a9cd-aedcf2bed54b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:45.819389 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 951/1914] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:33:45 2022] PUT /identity/v3/projects/df56de1ea1be4affa107cded85597809/users/9ea8d0d05b2a41429ccc77cc291a31e6/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:33:45.830532 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e8bd8-d879-4973-9905-fbe7001638b5 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:45.831295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e8bd8-d879-4973-9905-fbe7001638b5 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:45.831624 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e8bd8-d879-4973-9905-fbe7001638b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:45.866474 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-ea6e8bd8-d879-4973-9905-fbe7001638b5 None None] MFA Rules not processed for user `9ea8d0d05b2a41429ccc77cc291a31e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:45.867669 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-ea6e8bd8-d879-4973-9905-fbe7001638b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:45.993470 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 964/1915] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:46.006052 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64a785-a459-45d4-a9cd-aedcf2bed54b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:46.008483 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fb64a785-a459-45d4-a9cd-aedcf2bed54b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:46.114299 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64a785-a459-45d4-a9cd-aedcf2bed54b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:46.121239 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec122037-013f-4cb6-80f3-333f049e8320 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:46.122536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec122037-013f-4cb6-80f3-333f049e8320 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:46.122987 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec122037-013f-4cb6-80f3-333f049e8320 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:46.123655 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec122037-013f-4cb6-80f3-333f049e8320 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:46.135042 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec122037-013f-4cb6-80f3-333f049e8320 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:46.136509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec122037-013f-4cb6-80f3-333f049e8320 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:46.151349 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 952/1916] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:46.176672 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e8bd8-d879-4973-9905-fbe7001638b5 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:46.179106 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-ea6e8bd8-d879-4973-9905-fbe7001638b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:46.310552 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e8bd8-d879-4973-9905-fbe7001638b5 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:46.319804 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda6150-158b-41ea-a883-8efd1cc005b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:46.320228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda6150-158b-41ea-a883-8efd1cc005b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:46.320471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda6150-158b-41ea-a883-8efd1cc005b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:46.320756 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda6150-158b-41ea-a883-8efd1cc005b5 admin admin] PATH_INFO: `/v3/projects/006426cc90754abba6136e045277b67c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:46.322091 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda6150-158b-41ea-a883-8efd1cc005b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=006426cc90754abba6136e045277b67c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:46.326141 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda6150-158b-41ea-a883-8efd1cc005b5 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:46.444356 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-1eda6150-158b-41ea-a883-8efd1cc005b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '006426cc90754abba6136e045277b67c', 'request_id': 'req-1eda6150-158b-41ea-a883-8efd1cc005b5'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:46.459172 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-1eda6150-158b-41ea-a883-8efd1cc005b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '006426cc90754abba6136e045277b67c', 'request_id': 'req-1eda6150-158b-41ea-a883-8efd1cc005b5'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:46.470794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-1eda6150-158b-41ea-a883-8efd1cc005b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '006426cc90754abba6136e045277b67c', 'request_id': 'req-1eda6150-158b-41ea-a883-8efd1cc005b5'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:46.491673 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-1eda6150-158b-41ea-a883-8efd1cc005b5 admin admin] The token cache is being invalidate because project 006426cc90754abba6136e045277b67c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:33:46.493211 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-1eda6150-158b-41ea-a883-8efd1cc005b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:46.494150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-1eda6150-158b-41ea-a883-8efd1cc005b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:46.496634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 965/1917] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:33:46 2022] DELETE /identity/v3/projects/006426cc90754abba6136e045277b67c => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:46.501794 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d9f4e-b48b-4dfc-9b94-b81acbdfc38b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:46.502121 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d9f4e-b48b-4dfc-9b94-b81acbdfc38b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:46.502389 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d9f4e-b48b-4dfc-9b94-b81acbdfc38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:46.534135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ed2d9f4e-b48b-4dfc-9b94-b81acbdfc38b None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:46.535879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ed2d9f4e-b48b-4dfc-9b94-b81acbdfc38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:46.636023 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 953/1918] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:46 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:46.641582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205710e-1714-42e8-b3e9-339f785b5188 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:46.642174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205710e-1714-42e8-b3e9-339f785b5188 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:46.642267 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205710e-1714-42e8-b3e9-339f785b5188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:46.653499 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:46.675771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-a205710e-1714-42e8-b3e9-339f785b5188 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:46.690548 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-a205710e-1714-42e8-b3e9-339f785b5188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:46.713033 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 966/1919] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:46 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:46.717889 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d9f4e-b48b-4dfc-9b94-b81acbdfc38b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:46.733657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d9f4e-b48b-4dfc-9b94-b81acbdfc38b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:46.752786 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a673646-d920-4e02-8b44-a95f09ab458e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:46.756176 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a673646-d920-4e02-8b44-a95f09ab458e admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:46.756723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a673646-d920-4e02-8b44-a95f09ab458e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:46.760161 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a673646-d920-4e02-8b44-a95f09ab458e admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:46.767762 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a673646-d920-4e02-8b44-a95f09ab458e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:33:46.768556 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a673646-d920-4e02-8b44-a95f09ab458e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:46.775024 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a673646-d920-4e02-8b44-a95f09ab458e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:46.815034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 954/1920] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:33:46 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:33:46.819766 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205710e-1714-42e8-b3e9-339f785b5188 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:46.834566 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205710e-1714-42e8-b3e9-339f785b5188 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:46.851490 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229046e6-2019-4b87-9a94-0ad2000cc119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:46.855812 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229046e6-2019-4b87-9a94-0ad2000cc119 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:46.856198 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229046e6-2019-4b87-9a94-0ad2000cc119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:46.856518 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229046e6-2019-4b87-9a94-0ad2000cc119 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:46.857337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229046e6-2019-4b87-9a94-0ad2000cc119 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:46.859193 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229046e6-2019-4b87-9a94-0ad2000cc119 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:46.955316 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 967/1921] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:33:46 2022] POST /identity/v3/projects => generated 384 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:46.957006 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a673646-d920-4e02-8b44-a95f09ab458e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:46.986632 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a673646-d920-4e02-8b44-a95f09ab458e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:47.027803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:47.179320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e9d4b-545c-426a-bc0f-bff717ea7627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:47.180722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e9d4b-545c-426a-bc0f-bff717ea7627 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:47.181503 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e9d4b-545c-426a-bc0f-bff717ea7627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:47.182744 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e9d4b-545c-426a-bc0f-bff717ea7627 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:47.186968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e9d4b-545c-426a-bc0f-bff717ea7627 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:47.190534 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e9d4b-545c-426a-bc0f-bff717ea7627 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:47.235899 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 955/1922] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:33:46 2022] POST /identity/v3/users => generated 293 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:47.241266 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229046e6-2019-4b87-9a94-0ad2000cc119 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:47.257865 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229046e6-2019-4b87-9a94-0ad2000cc119 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:47.271442 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226b4e8-8468-496b-a968-f3544083a798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:47.272275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226b4e8-8468-496b-a968-f3544083a798 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:47.272661 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226b4e8-8468-496b-a968-f3544083a798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:47.273039 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226b4e8-8468-496b-a968-f3544083a798 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:47.277056 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2226b4e8-8468-496b-a968-f3544083a798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:47.278937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2226b4e8-8468-496b-a968-f3544083a798 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:47.301502 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 968/1923] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:47 2022] GET /identity/v3/roles => generated 1427 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:47.305988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e9d4b-545c-426a-bc0f-bff717ea7627 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:47.321474 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e9d4b-545c-426a-bc0f-bff717ea7627 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:47.350917 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3542608-8a95-44d3-8710-3cb7744e0837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:47.351738 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3542608-8a95-44d3-8710-3cb7744e0837 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:47.351885 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3542608-8a95-44d3-8710-3cb7744e0837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:47.352168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3542608-8a95-44d3-8710-3cb7744e0837 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:47.353570 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3542608-8a95-44d3-8710-3cb7744e0837 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:47.356416 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3542608-8a95-44d3-8710-3cb7744e0837 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:47.387722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 956/1924] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:47 2022] GET /identity/v3/roles => generated 1427 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:47.392346 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226b4e8-8468-496b-a968-f3544083a798 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:47.416841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226b4e8-8468-496b-a968-f3544083a798 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:47.444375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29cc1f1-f3bd-4c0a-9c4c-919858799425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:47.445182 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cc1f1-f3bd-4c0a-9c4c-919858799425 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:47.445575 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cc1f1-f3bd-4c0a-9c4c-919858799425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:47.445963 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cc1f1-f3bd-4c0a-9c4c-919858799425 admin admin] PATH_INFO: `/v3/projects/f9c972cb2ce14cb5bd9b9e29d3000b2a/users/acea7f13135346ea9b5ce1d06abb4491/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:47.454999 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e155489-f5c5-45d5-a451-4f02fe2e5b24 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:47.455675 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e155489-f5c5-45d5-a451-4f02fe2e5b24 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:47.456158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e155489-f5c5-45d5-a451-4f02fe2e5b24 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:47.458298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 957/1925] 10.222.0.121 () {54 vars in 926 bytes} [Thu Aug 4 10:33:47 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:33:47.469215 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254e4c2-59ef-419b-8009-e62af7d40d4c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:47.469935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254e4c2-59ef-419b-8009-e62af7d40d4c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:47.470369 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254e4c2-59ef-419b-8009-e62af7d40d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:47.488575 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29cc1f1-f3bd-4c0a-9c4c-919858799425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9c972cb2ce14cb5bd9b9e29d3000b2a, user_id=acea7f13135346ea9b5ce1d06abb4491, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:47.491511 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29cc1f1-f3bd-4c0a-9c4c-919858799425 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:47.515268 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2254e4c2-59ef-419b-8009-e62af7d40d4c None None] MFA Rules not processed for user `0d947aeb749546309a8d4b7938087ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:47.516703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2254e4c2-59ef-419b-8009-e62af7d40d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:47.528523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 969/1926] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:33:47 2022] PUT /identity/v3/projects/f9c972cb2ce14cb5bd9b9e29d3000b2a/users/acea7f13135346ea9b5ce1d06abb4491/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:47.542328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c33c0-e263-4f96-8efd-56a5fd39b716 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:47.544745 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c33c0-e263-4f96-8efd-56a5fd39b716 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:47.545243 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c33c0-e263-4f96-8efd-56a5fd39b716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:47.624239 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-878c33c0-e263-4f96-8efd-56a5fd39b716 None None] MFA Rules not processed for user `acea7f13135346ea9b5ce1d06abb4491`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:47.626140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-878c33c0-e263-4f96-8efd-56a5fd39b716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:47.717917 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 958/1927] 10.222.0.121 () {58 vars in 1023 bytes} [Thu Aug 4 10:33:47 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:33:47.778766 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 970/1928] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:47 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:47.783849 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254e4c2-59ef-419b-8009-e62af7d40d4c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:47.879836 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c33c0-e263-4f96-8efd-56a5fd39b716 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:47.888015 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-878c33c0-e263-4f96-8efd-56a5fd39b716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:47.902065 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254e4c2-59ef-419b-8009-e62af7d40d4c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:47.911151 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b87be-127e-43a3-bae9-5a50a2932794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:47.912134 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b87be-127e-43a3-bae9-5a50a2932794 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:47.912638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b87be-127e-43a3-bae9-5a50a2932794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:47.913150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b87be-127e-43a3-bae9-5a50a2932794 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:47.914064 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b87be-127e-43a3-bae9-5a50a2932794 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:47.916865 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b87be-127e-43a3-bae9-5a50a2932794 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:47.921482 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c33c0-e263-4f96-8efd-56a5fd39b716 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:47.934787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562d3b8-8006-421c-bfd5-d6d8f6c75e93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:47.935817 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562d3b8-8006-421c-bfd5-d6d8f6c75e93 service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:47.936323 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562d3b8-8006-421c-bfd5-d6d8f6c75e93 service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:47.936715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562d3b8-8006-421c-bfd5-d6d8f6c75e93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:47.955727 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562d3b8-8006-421c-bfd5-d6d8f6c75e93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:47.958759 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562d3b8-8006-421c-bfd5-d6d8f6c75e93 service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:47.969581 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 959/1929] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:33:47 2022] POST /identity/v3/projects => generated 386 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:47.977018 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b87be-127e-43a3-bae9-5a50a2932794 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:47.986277 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 971/1930] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:47 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:48.010426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:48.129888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b87be-127e-43a3-bae9-5a50a2932794 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:48.142276 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294b997-4560-4eb1-8a19-c95ac1b9d2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:48.143553 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294b997-4560-4eb1-8a19-c95ac1b9d2d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:48.144041 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294b997-4560-4eb1-8a19-c95ac1b9d2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:48.144633 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294b997-4560-4eb1-8a19-c95ac1b9d2d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:48.145538 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294b997-4560-4eb1-8a19-c95ac1b9d2d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:48.147759 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294b997-4560-4eb1-8a19-c95ac1b9d2d8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:48.189146 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 960/1931] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:33:47 2022] POST /identity/v3/users => generated 300 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:48.193250 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562d3b8-8006-421c-bfd5-d6d8f6c75e93 service nova] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:48.214534 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562d3b8-8006-421c-bfd5-d6d8f6c75e93 service nova] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:48.232441 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddbe02-4663-4cce-a63b-55fffa9c023f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:48.234248 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddbe02-4663-4cce-a63b-55fffa9c023f admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:48.235040 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddbe02-4663-4cce-a63b-55fffa9c023f admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:48.235737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddbe02-4663-4cce-a63b-55fffa9c023f admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:48.236874 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddbe02-4663-4cce-a63b-55fffa9c023f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:48.238889 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddbe02-4663-4cce-a63b-55fffa9c023f admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:48.270819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 972/1932] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:48 2022] GET /identity/v3/roles => generated 1427 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:33:48.275191 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294b997-4560-4eb1-8a19-c95ac1b9d2d8 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:48.288868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294b997-4560-4eb1-8a19-c95ac1b9d2d8 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:48.304865 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7c10f-8dfa-4c6b-ad7d-1a3659b318d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:48.305907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7c10f-8dfa-4c6b-ad7d-1a3659b318d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:48.306688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7c10f-8dfa-4c6b-ad7d-1a3659b318d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:48.307024 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7c10f-8dfa-4c6b-ad7d-1a3659b318d7 admin admin] PATH_INFO: `/v3/projects/8f50765ec5854049bafcb12b33d24593/users/31b4182e6beb438e8e881e699ef2ef59/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:48.332533 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7c10f-8dfa-4c6b-ad7d-1a3659b318d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f50765ec5854049bafcb12b33d24593, user_id=31b4182e6beb438e8e881e699ef2ef59, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:48.335315 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7c10f-8dfa-4c6b-ad7d-1a3659b318d7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:48.355152 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 961/1933] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:33:48 2022] PUT /identity/v3/projects/8f50765ec5854049bafcb12b33d24593/users/31b4182e6beb438e8e881e699ef2ef59/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:48.359736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddbe02-4663-4cce-a63b-55fffa9c023f admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:48.372522 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddbe02-4663-4cce-a63b-55fffa9c023f admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:48.455533 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baffef15-7f4c-4e6d-8558-6990fcef18cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:48.456236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffef15-7f4c-4e6d-8558-6990fcef18cd admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:48.456632 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffef15-7f4c-4e6d-8558-6990fcef18cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:48.457156 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffef15-7f4c-4e6d-8558-6990fcef18cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:48.457892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baffef15-7f4c-4e6d-8558-6990fcef18cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:48.459844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baffef15-7f4c-4e6d-8558-6990fcef18cd admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:48.472616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 973/1934] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:48 2022] GET /identity/v3/roles => generated 1427 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:48.477210 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7c10f-8dfa-4c6b-ad7d-1a3659b318d7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:48.489817 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7c10f-8dfa-4c6b-ad7d-1a3659b318d7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:48.497947 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4a883-404e-40a5-bc1b-f6fa03a03519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:48.498767 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4a883-404e-40a5-bc1b-f6fa03a03519 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:48.499050 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4a883-404e-40a5-bc1b-f6fa03a03519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:48.499339 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4a883-404e-40a5-bc1b-f6fa03a03519 admin admin] PATH_INFO: `/v3/projects/8f50765ec5854049bafcb12b33d24593/users/31b4182e6beb438e8e881e699ef2ef59/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:48.502153 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4a883-404e-40a5-bc1b-f6fa03a03519 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f50765ec5854049bafcb12b33d24593, user_id=31b4182e6beb438e8e881e699ef2ef59, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:48.504899 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4a883-404e-40a5-bc1b-f6fa03a03519 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:48.518073 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 962/1935] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:33:48 2022] PUT /identity/v3/projects/8f50765ec5854049bafcb12b33d24593/users/31b4182e6beb438e8e881e699ef2ef59/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:48.529817 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce39d7-0341-4736-ba06-cecf0bcd0b8f None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:48.530114 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce39d7-0341-4736-ba06-cecf0bcd0b8f None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:48.530401 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce39d7-0341-4736-ba06-cecf0bcd0b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:48.538776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:48.566207 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-29ce39d7-0341-4736-ba06-cecf0bcd0b8f None None] MFA Rules not processed for user `31b4182e6beb438e8e881e699ef2ef59`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:48.567481 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-29ce39d7-0341-4736-ba06-cecf0bcd0b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:48.643024 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 974/1936] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:48 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:48.657184 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4a883-404e-40a5-bc1b-f6fa03a03519 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:48.659546 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8de4a883-404e-40a5-bc1b-f6fa03a03519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:48.728652 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4a883-404e-40a5-bc1b-f6fa03a03519 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:48.732723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2893bf-c343-400a-a6ca-83503f7e47c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:48.733351 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2893bf-c343-400a-a6ca-83503f7e47c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:48.733638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2893bf-c343-400a-a6ca-83503f7e47c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:48.733916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2893bf-c343-400a-a6ca-83503f7e47c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:48.803464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2893bf-c343-400a-a6ca-83503f7e47c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:48.804944 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2893bf-c343-400a-a6ca-83503f7e47c2 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:48.814614 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 963/1937] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:48 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:50.126861 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce735d5-61b1-460b-9c8c-fc93998db05e None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:50.127451 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce735d5-61b1-460b-9c8c-fc93998db05e None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:50.127883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce735d5-61b1-460b-9c8c-fc93998db05e None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:50.129698 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 975/1938] 10.222.0.121 () {54 vars in 926 bytes} [Thu Aug 4 10:33:50 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:33:50.138939 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda2f19-11c4-4e57-a9fc-679c644d738e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:50.139517 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda2f19-11c4-4e57-a9fc-679c644d738e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:50.139517 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda2f19-11c4-4e57-a9fc-679c644d738e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:50.167983 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6eda2f19-11c4-4e57-a9fc-679c644d738e None None] MFA Rules not processed for user `01c7cfbe911c4378857848966f4f8d98`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:50.169581 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6eda2f19-11c4-4e57-a9fc-679c644d738e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:50.208253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:50.302513 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 964/1939] 10.222.0.121 () {58 vars in 1023 bytes} [Thu Aug 4 10:33:50 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:50.325032 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce735d5-61b1-460b-9c8c-fc93998db05e None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:50.327198 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-1ce735d5-61b1-460b-9c8c-fc93998db05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:50.348735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce735d5-61b1-460b-9c8c-fc93998db05e None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:50.356361 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d394940-96ca-40a4-8c28-136e8e948d78 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c7cfbe911c4378857848966f4f8d98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:50.357109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d394940-96ca-40a4-8c28-136e8e948d78 service placement] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:50.358679 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d394940-96ca-40a4-8c28-136e8e948d78 service placement] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:50.358679 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d394940-96ca-40a4-8c28-136e8e948d78 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:50.369623 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d394940-96ca-40a4-8c28-136e8e948d78 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:50.372083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d394940-96ca-40a4-8c28-136e8e948d78 service placement] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:50.388013 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 976/1940] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:50 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:33:50.820581 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda2f19-11c4-4e57-a9fc-679c644d738e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:50.822349 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6eda2f19-11c4-4e57-a9fc-679c644d738e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:50.898625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda2f19-11c4-4e57-a9fc-679c644d738e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:50.904466 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b2951-98b5-4ce9-99f7-2be7710cc6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:50.905197 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b2951-98b5-4ce9-99f7-2be7710cc6bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:50.905475 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b2951-98b5-4ce9-99f7-2be7710cc6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:50.905756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b2951-98b5-4ce9-99f7-2be7710cc6bf admin admin] PATH_INFO: `/v3/users/e0984ca333794b0d9b04e61fe13c2485` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:50.907084 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b2951-98b5-4ce9-99f7-2be7710cc6bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0984ca333794b0d9b04e61fe13c2485)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:50.909104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b2951-98b5-4ce9-99f7-2be7710cc6bf admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:50.960047 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ae5b2951-98b5-4ce9-99f7-2be7710cc6bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0984ca333794b0d9b04e61fe13c2485'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:50.967745 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ae5b2951-98b5-4ce9-99f7-2be7710cc6bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0984ca333794b0d9b04e61fe13c2485'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:50.967881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ae5b2951-98b5-4ce9-99f7-2be7710cc6bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0984ca333794b0d9b04e61fe13c2485'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:50.981239 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ae5b2951-98b5-4ce9-99f7-2be7710cc6bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0984ca333794b0d9b04e61fe13c2485'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:50.981964 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ae5b2951-98b5-4ce9-99f7-2be7710cc6bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0984ca333794b0d9b04e61fe13c2485'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:50.998394 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 965/1941] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:33:50 2022] DELETE /identity/v3/users/e0984ca333794b0d9b04e61fe13c2485 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:51.186886 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d394940-96ca-40a4-8c28-136e8e948d78 service placement] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:51.202929 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d394940-96ca-40a4-8c28-136e8e948d78 service placement] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:51.263036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973da3e7-4dea-497f-8cef-c93736dd890f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:51.263700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973da3e7-4dea-497f-8cef-c93736dd890f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:51.264012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973da3e7-4dea-497f-8cef-c93736dd890f admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:51.264273 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973da3e7-4dea-497f-8cef-c93736dd890f admin admin] PATH_INFO: `/v3/projects/cd8d7f51f7ab4e0f8a86effcc1643dfe` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:51.265543 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973da3e7-4dea-497f-8cef-c93736dd890f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd8d7f51f7ab4e0f8a86effcc1643dfe)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:51.267498 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973da3e7-4dea-497f-8cef-c93736dd890f admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:51.317893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-973da3e7-4dea-497f-8cef-c93736dd890f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd8d7f51f7ab4e0f8a86effcc1643dfe', 'request_id': 'req-973da3e7-4dea-497f-8cef-c93736dd890f'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:51.326300 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-973da3e7-4dea-497f-8cef-c93736dd890f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd8d7f51f7ab4e0f8a86effcc1643dfe', 'request_id': 'req-973da3e7-4dea-497f-8cef-c93736dd890f'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:51.332707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-973da3e7-4dea-497f-8cef-c93736dd890f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd8d7f51f7ab4e0f8a86effcc1643dfe', 'request_id': 'req-973da3e7-4dea-497f-8cef-c93736dd890f'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:51.345407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-973da3e7-4dea-497f-8cef-c93736dd890f admin admin] The token cache is being invalidate because project cd8d7f51f7ab4e0f8a86effcc1643dfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:33:51.345722 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-973da3e7-4dea-497f-8cef-c93736dd890f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:51.346502 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-973da3e7-4dea-497f-8cef-c93736dd890f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:51.348200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 977/1942] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:33:51 2022] DELETE /identity/v3/projects/cd8d7f51f7ab4e0f8a86effcc1643dfe => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:51.352814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b2951-98b5-4ce9-99f7-2be7710cc6bf admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:51.355337 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ae5b2951-98b5-4ce9-99f7-2be7710cc6bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:51.369558 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b2951-98b5-4ce9-99f7-2be7710cc6bf admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:51.408954 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:51.436663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53073044-a907-4866-9296-243279ce1c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:51.437426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53073044-a907-4866-9296-243279ce1c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:51.437854 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53073044-a907-4866-9296-243279ce1c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:51.438273 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53073044-a907-4866-9296-243279ce1c1c admin admin] PATH_INFO: `/v3/users/a0d4f49c68d3442f89c33b3f7f9fd9aa` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:51.439513 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53073044-a907-4866-9296-243279ce1c1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0d4f49c68d3442f89c33b3f7f9fd9aa)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:51.441620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53073044-a907-4866-9296-243279ce1c1c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:51.502040 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-53073044-a907-4866-9296-243279ce1c1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0d4f49c68d3442f89c33b3f7f9fd9aa'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:51.510414 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-53073044-a907-4866-9296-243279ce1c1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0d4f49c68d3442f89c33b3f7f9fd9aa'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:51.511840 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-53073044-a907-4866-9296-243279ce1c1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0d4f49c68d3442f89c33b3f7f9fd9aa'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:51.527074 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-53073044-a907-4866-9296-243279ce1c1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0d4f49c68d3442f89c33b3f7f9fd9aa'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:51.527845 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-53073044-a907-4866-9296-243279ce1c1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0d4f49c68d3442f89c33b3f7f9fd9aa'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:51.550250 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 966/1943] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:33:51 2022] DELETE /identity/v3/users/a0d4f49c68d3442f89c33b3f7f9fd9aa => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:51.752625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973da3e7-4dea-497f-8cef-c93736dd890f admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:51.754356 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-973da3e7-4dea-497f-8cef-c93736dd890f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:51.764657 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:51.771492 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973da3e7-4dea-497f-8cef-c93736dd890f admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:51.844270 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a27ae-e0ba-4bf9-b6a0-b1909de92777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:51.845090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a27ae-e0ba-4bf9-b6a0-b1909de92777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:51.845445 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a27ae-e0ba-4bf9-b6a0-b1909de92777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:51.845796 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a27ae-e0ba-4bf9-b6a0-b1909de92777 admin admin] PATH_INFO: `/v3/projects/c99bb9fb09664e279cf4450ad416b576` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:51.847213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a27ae-e0ba-4bf9-b6a0-b1909de92777 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c99bb9fb09664e279cf4450ad416b576)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:51.849387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a27ae-e0ba-4bf9-b6a0-b1909de92777 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:51.922130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0e9a27ae-e0ba-4bf9-b6a0-b1909de92777 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c99bb9fb09664e279cf4450ad416b576', 'request_id': 'req-0e9a27ae-e0ba-4bf9-b6a0-b1909de92777'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:51.933320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0e9a27ae-e0ba-4bf9-b6a0-b1909de92777 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c99bb9fb09664e279cf4450ad416b576', 'request_id': 'req-0e9a27ae-e0ba-4bf9-b6a0-b1909de92777'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:51.944261 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0e9a27ae-e0ba-4bf9-b6a0-b1909de92777 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c99bb9fb09664e279cf4450ad416b576', 'request_id': 'req-0e9a27ae-e0ba-4bf9-b6a0-b1909de92777'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:51.955495 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0e9a27ae-e0ba-4bf9-b6a0-b1909de92777 admin admin] The token cache is being invalidate because project c99bb9fb09664e279cf4450ad416b576 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:33:51.955790 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0e9a27ae-e0ba-4bf9-b6a0-b1909de92777 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:51.956522 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0e9a27ae-e0ba-4bf9-b6a0-b1909de92777 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:33:51.958112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 978/1944] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:33:51 2022] DELETE /identity/v3/projects/c99bb9fb09664e279cf4450ad416b576 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:51.963838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90c3c8-18bb-4f8b-9533-36d474442eaf None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:51.964158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90c3c8-18bb-4f8b-9533-36d474442eaf None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:51.964436 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90c3c8-18bb-4f8b-9533-36d474442eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:51.986308 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-fd90c3c8-18bb-4f8b-9533-36d474442eaf None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:51.987831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fd90c3c8-18bb-4f8b-9533-36d474442eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:52.061151 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 967/1945] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:33:52.070061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db64486-9658-4082-a917-aec43f496e57 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:52.070349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db64486-9658-4082-a917-aec43f496e57 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:52.070636 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db64486-9658-4082-a917-aec43f496e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:52.092902 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-4db64486-9658-4082-a917-aec43f496e57 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:52.094225 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-4db64486-9658-4082-a917-aec43f496e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:52.103191 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 979/1946] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:52.108009 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd90c3c8-18bb-4f8b-9533-36d474442eaf None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:52.118936 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd90c3c8-18bb-4f8b-9533-36d474442eaf None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:52.123801 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17a600-7c91-4cd2-921d-d716f6cb104a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:52.124471 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17a600-7c91-4cd2-921d-d716f6cb104a admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:52.124750 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17a600-7c91-4cd2-921d-d716f6cb104a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:52.125026 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17a600-7c91-4cd2-921d-d716f6cb104a admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:52.125814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e17a600-7c91-4cd2-921d-d716f6cb104a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:33:52.126120 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e17a600-7c91-4cd2-921d-d716f6cb104a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:52.127733 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e17a600-7c91-4cd2-921d-d716f6cb104a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:52.140776 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 968/1947] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:33:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:33:52.145550 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db64486-9658-4082-a917-aec43f496e57 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:52.155704 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db64486-9658-4082-a917-aec43f496e57 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:52.162125 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a7718-6208-47ba-848f-6c4858fb1805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:52.162901 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a7718-6208-47ba-848f-6c4858fb1805 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:52.163254 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a7718-6208-47ba-848f-6c4858fb1805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:52.163518 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a7718-6208-47ba-848f-6c4858fb1805 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:52.164423 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a7718-6208-47ba-848f-6c4858fb1805 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:52.166712 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a7718-6208-47ba-848f-6c4858fb1805 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:52.200105 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 980/1948] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:33:52 2022] POST /identity/v3/projects => generated 376 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:33:52.208277 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17a600-7c91-4cd2-921d-d716f6cb104a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:52.220085 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17a600-7c91-4cd2-921d-d716f6cb104a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:52.269594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:52.287245 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b863e-c030-4e7a-9059-761f8b1a9976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:52.288123 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b863e-c030-4e7a-9059-761f8b1a9976 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:52.288417 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b863e-c030-4e7a-9059-761f8b1a9976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:52.288702 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b863e-c030-4e7a-9059-761f8b1a9976 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:52.289667 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b863e-c030-4e7a-9059-761f8b1a9976 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:52.291380 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b863e-c030-4e7a-9059-761f8b1a9976 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:52.316991 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 969/1949] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:33:52 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:33:52.321531 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a7718-6208-47ba-848f-6c4858fb1805 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:52.334258 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a7718-6208-47ba-848f-6c4858fb1805 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:52.340183 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e240166-248d-4bec-a79e-a021a19ce2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:52.340840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e240166-248d-4bec-a79e-a021a19ce2c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:52.341129 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e240166-248d-4bec-a79e-a021a19ce2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:52.341415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e240166-248d-4bec-a79e-a021a19ce2c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:52.342131 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e240166-248d-4bec-a79e-a021a19ce2c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:52.344229 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e240166-248d-4bec-a79e-a021a19ce2c6 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:52.356417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 981/1950] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:52 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:52.361313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b863e-c030-4e7a-9059-761f8b1a9976 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:52.373833 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b863e-c030-4e7a-9059-761f8b1a9976 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:52.380675 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016d59f-5938-4d56-8902-c3c3b47f348e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:52.380915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016d59f-5938-4d56-8902-c3c3b47f348e admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:52.381390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016d59f-5938-4d56-8902-c3c3b47f348e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:52.381673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016d59f-5938-4d56-8902-c3c3b47f348e admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:52.382409 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016d59f-5938-4d56-8902-c3c3b47f348e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:52.384124 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016d59f-5938-4d56-8902-c3c3b47f348e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:52.396575 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 970/1951] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:33:52 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:33:52.410944 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e240166-248d-4bec-a79e-a021a19ce2c6 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:52.412246 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e240166-248d-4bec-a79e-a021a19ce2c6 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:52.418779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ef187-023c-4d5f-8f33-19238dcd8bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:52.419508 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ef187-023c-4d5f-8f33-19238dcd8bcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:52.419869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ef187-023c-4d5f-8f33-19238dcd8bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:52.420297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ef187-023c-4d5f-8f33-19238dcd8bcf admin admin] PATH_INFO: `/v3/projects/2439b61aa6b7466288b1e4b03a87c5e8/users/e204358f8e714395b14b51419eadc071/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:52.442550 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ef187-023c-4d5f-8f33-19238dcd8bcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2439b61aa6b7466288b1e4b03a87c5e8, user_id=e204358f8e714395b14b51419eadc071, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:52.446474 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ef187-023c-4d5f-8f33-19238dcd8bcf admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:52.463522 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 982/1952] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:33:52 2022] PUT /identity/v3/projects/2439b61aa6b7466288b1e4b03a87c5e8/users/e204358f8e714395b14b51419eadc071/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:33:52.474545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca34447-95ff-4703-9c09-0a463e685404 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:52.474850 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca34447-95ff-4703-9c09-0a463e685404 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:52.475124 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca34447-95ff-4703-9c09-0a463e685404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:52.509823 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-bca34447-95ff-4703-9c09-0a463e685404 None None] MFA Rules not processed for user `e204358f8e714395b14b51419eadc071`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:33:52.511088 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bca34447-95ff-4703-9c09-0a463e685404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:52.612369 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 971/1953] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:33:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:33:52.625011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ef187-023c-4d5f-8f33-19238dcd8bcf admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:33:52.627453 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-fa8ef187-023c-4d5f-8f33-19238dcd8bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:33:52.673251 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:33:52.731744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ef187-023c-4d5f-8f33-19238dcd8bcf admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:33:52.736083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076dab4-589e-4217-8ae2-cf4c6c1f7c82 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b31c86bc9334d2b8acc35a035f90442', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:33:52.736749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076dab4-589e-4217-8ae2-cf4c6c1f7c82 service glance] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:33:52.737034 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076dab4-589e-4217-8ae2-cf4c6c1f7c82 service glance] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:33:52.737307 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076dab4-589e-4217-8ae2-cf4c6c1f7c82 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:33:52.749383 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076dab4-589e-4217-8ae2-cf4c6c1f7c82 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:33:52.750978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076dab4-589e-4217-8ae2-cf4c6c1f7c82 service glance] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:33:52.761172 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 983/1954] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:33:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:02.607296 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=46,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:02.756756 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=21 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:10.309209 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca34447-95ff-4703-9c09-0a463e685404 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:10.310928 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bca34447-95ff-4703-9c09-0a463e685404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:10.374877 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca34447-95ff-4703-9c09-0a463e685404 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:10.379216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e8e31-9e40-4b44-afcd-d7316bc0b9e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:10.379855 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e8e31-9e40-4b44-afcd-d7316bc0b9e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:10.380195 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e8e31-9e40-4b44-afcd-d7316bc0b9e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:10.380473 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e8e31-9e40-4b44-afcd-d7316bc0b9e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:10.382385 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6a7e8e31-9e40-4b44-afcd-d7316bc0b9e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:10.454851 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e8e31-9e40-4b44-afcd-d7316bc0b9e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:10.456262 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e8e31-9e40-4b44-afcd-d7316bc0b9e1 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:10.466138 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 972/1955] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:34:10 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:11.046793 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076dab4-589e-4217-8ae2-cf4c6c1f7c82 service glance] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:11.048320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d076dab4-589e-4217-8ae2-cf4c6c1f7c82 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:11.123301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076dab4-589e-4217-8ae2-cf4c6c1f7c82 service glance] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:11.127905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5dad27-a8e9-43e2-8da7-4d8acf678ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:11.128593 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5dad27-a8e9-43e2-8da7-4d8acf678ad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:11.128876 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5dad27-a8e9-43e2-8da7-4d8acf678ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:11.129152 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5dad27-a8e9-43e2-8da7-4d8acf678ad5 admin admin] PATH_INFO: `/v3/users/9ae85e53631f4de8b254cefb428ee229` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:11.130322 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5dad27-a8e9-43e2-8da7-4d8acf678ad5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ae85e53631f4de8b254cefb428ee229)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:11.132268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5dad27-a8e9-43e2-8da7-4d8acf678ad5 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:11.189941 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-ba5dad27-a8e9-43e2-8da7-4d8acf678ad5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ae85e53631f4de8b254cefb428ee229'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:11.190695 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-ba5dad27-a8e9-43e2-8da7-4d8acf678ad5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ae85e53631f4de8b254cefb428ee229'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:11.203621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-ba5dad27-a8e9-43e2-8da7-4d8acf678ad5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ae85e53631f4de8b254cefb428ee229'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:11.210799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-ba5dad27-a8e9-43e2-8da7-4d8acf678ad5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ae85e53631f4de8b254cefb428ee229'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:11.222275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-ba5dad27-a8e9-43e2-8da7-4d8acf678ad5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ae85e53631f4de8b254cefb428ee229'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:11.230439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 984/1956] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:34:11 2022] DELETE /identity/v3/users/9ae85e53631f4de8b254cefb428ee229 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:11.235714 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a79737-5b1f-4e4c-933b-483a966a7b66 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:11.236042 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a79737-5b1f-4e4c-933b-483a966a7b66 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:11.236329 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a79737-5b1f-4e4c-933b-483a966a7b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:11.256515 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-60a79737-5b1f-4e4c-933b-483a966a7b66 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:11.257757 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-60a79737-5b1f-4e4c-933b-483a966a7b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:11.328503 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 973/1957] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:11 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:11.340823 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5dad27-a8e9-43e2-8da7-4d8acf678ad5 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:11.342604 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-ba5dad27-a8e9-43e2-8da7-4d8acf678ad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:11.427242 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5dad27-a8e9-43e2-8da7-4d8acf678ad5 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:11.431740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6982365-9f33-4082-bf6c-7d11c7540259 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:11.432459 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6982365-9f33-4082-bf6c-7d11c7540259 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:11.432817 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6982365-9f33-4082-bf6c-7d11c7540259 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:11.433022 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6982365-9f33-4082-bf6c-7d11c7540259 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:11.444214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6982365-9f33-4082-bf6c-7d11c7540259 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:11.445686 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6982365-9f33-4082-bf6c-7d11c7540259 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:11.455977 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 985/1958] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:34:11 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:11.773995 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a79737-5b1f-4e4c-933b-483a966a7b66 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:11.776101 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-60a79737-5b1f-4e4c-933b-483a966a7b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:11.790818 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a79737-5b1f-4e4c-933b-483a966a7b66 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:11.797935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29187c60-304e-4b04-86f7-75a9a5e4bd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:11.799351 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29187c60-304e-4b04-86f7-75a9a5e4bd2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:11.799727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29187c60-304e-4b04-86f7-75a9a5e4bd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:11.800400 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29187c60-304e-4b04-86f7-75a9a5e4bd2a admin admin] PATH_INFO: `/v3/projects/cc2a4e4a15c741978625363534446549` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:11.802207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29187c60-304e-4b04-86f7-75a9a5e4bd2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc2a4e4a15c741978625363534446549)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:11.805000 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29187c60-304e-4b04-86f7-75a9a5e4bd2a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:11.924150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:11.934786 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-29187c60-304e-4b04-86f7-75a9a5e4bd2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc2a4e4a15c741978625363534446549', 'request_id': 'req-29187c60-304e-4b04-86f7-75a9a5e4bd2a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:11.947829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-29187c60-304e-4b04-86f7-75a9a5e4bd2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc2a4e4a15c741978625363534446549', 'request_id': 'req-29187c60-304e-4b04-86f7-75a9a5e4bd2a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:11.957549 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-29187c60-304e-4b04-86f7-75a9a5e4bd2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc2a4e4a15c741978625363534446549', 'request_id': 'req-29187c60-304e-4b04-86f7-75a9a5e4bd2a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:11.976799 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-29187c60-304e-4b04-86f7-75a9a5e4bd2a admin admin] The token cache is being invalidate because project cc2a4e4a15c741978625363534446549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:34:11.977102 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-29187c60-304e-4b04-86f7-75a9a5e4bd2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:11.978119 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-29187c60-304e-4b04-86f7-75a9a5e4bd2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:11.979711 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 974/1959] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:34:11 2022] DELETE /identity/v3/projects/cc2a4e4a15c741978625363534446549 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:11.984714 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6982365-9f33-4082-bf6c-7d11c7540259 service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:11.996321 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6982365-9f33-4082-bf6c-7d11c7540259 service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:12.032760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:12.111687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17053e74-af30-4590-b9fd-6df044375e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:12.112471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17053e74-af30-4590-b9fd-6df044375e39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:12.112678 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17053e74-af30-4590-b9fd-6df044375e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:12.112954 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17053e74-af30-4590-b9fd-6df044375e39 admin admin] PATH_INFO: `/v3/users/143954d97cfa491cbd467f346e51e184` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:12.115008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17053e74-af30-4590-b9fd-6df044375e39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=143954d97cfa491cbd467f346e51e184)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:12.118460 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17053e74-af30-4590-b9fd-6df044375e39 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:12.229992 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-17053e74-af30-4590-b9fd-6df044375e39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '143954d97cfa491cbd467f346e51e184'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:12.237716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-17053e74-af30-4590-b9fd-6df044375e39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '143954d97cfa491cbd467f346e51e184'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:12.270774 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-17053e74-af30-4590-b9fd-6df044375e39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '143954d97cfa491cbd467f346e51e184'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:12.289074 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-17053e74-af30-4590-b9fd-6df044375e39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '143954d97cfa491cbd467f346e51e184'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:12.307519 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-17053e74-af30-4590-b9fd-6df044375e39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '143954d97cfa491cbd467f346e51e184'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:12.316834 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 986/1960] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:34:11 2022] DELETE /identity/v3/users/143954d97cfa491cbd467f346e51e184 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:12.899906 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29187c60-304e-4b04-86f7-75a9a5e4bd2a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:12.901213 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-29187c60-304e-4b04-86f7-75a9a5e4bd2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:12.923412 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29187c60-304e-4b04-86f7-75a9a5e4bd2a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:12.997462 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e736ab8b-5fab-4465-81b5-1ba4bf485842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:12.998293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736ab8b-5fab-4465-81b5-1ba4bf485842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:12.998689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736ab8b-5fab-4465-81b5-1ba4bf485842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:12.999035 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736ab8b-5fab-4465-81b5-1ba4bf485842 admin admin] PATH_INFO: `/v3/projects/89cd83fc7f994298a562b2a66b7e4f5e` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:13.000438 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e736ab8b-5fab-4465-81b5-1ba4bf485842 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89cd83fc7f994298a562b2a66b7e4f5e)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:13.002396 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e736ab8b-5fab-4465-81b5-1ba4bf485842 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:13.063807 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e736ab8b-5fab-4465-81b5-1ba4bf485842 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89cd83fc7f994298a562b2a66b7e4f5e', 'request_id': 'req-e736ab8b-5fab-4465-81b5-1ba4bf485842'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:13.072997 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e736ab8b-5fab-4465-81b5-1ba4bf485842 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89cd83fc7f994298a562b2a66b7e4f5e', 'request_id': 'req-e736ab8b-5fab-4465-81b5-1ba4bf485842'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:13.083230 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e736ab8b-5fab-4465-81b5-1ba4bf485842 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89cd83fc7f994298a562b2a66b7e4f5e', 'request_id': 'req-e736ab8b-5fab-4465-81b5-1ba4bf485842'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:13.099397 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e736ab8b-5fab-4465-81b5-1ba4bf485842 admin admin] The token cache is being invalidate because project 89cd83fc7f994298a562b2a66b7e4f5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:34:13.099696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e736ab8b-5fab-4465-81b5-1ba4bf485842 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:13.100450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e736ab8b-5fab-4465-81b5-1ba4bf485842 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:13.102630 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 975/1961] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:34:12 2022] DELETE /identity/v3/projects/89cd83fc7f994298a562b2a66b7e4f5e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:13.108999 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9461b-6a45-44f7-b0b9-b2a22cd18c2e None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:13.109347 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9461b-6a45-44f7-b0b9-b2a22cd18c2e None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:13.109669 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9461b-6a45-44f7-b0b9-b2a22cd18c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:13.131425 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-b5b9461b-6a45-44f7-b0b9-b2a22cd18c2e None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:13.132700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-b5b9461b-6a45-44f7-b0b9-b2a22cd18c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:13.222291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 987/1962] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:13.233023 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e11b3-1fde-4de0-904b-b533005feffe None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:13.233310 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e11b3-1fde-4de0-904b-b533005feffe None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:13.233583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e11b3-1fde-4de0-904b-b533005feffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:13.256187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a08e11b3-1fde-4de0-904b-b533005feffe None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:13.257472 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a08e11b3-1fde-4de0-904b-b533005feffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:13.266522 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 976/1963] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:13.272292 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9461b-6a45-44f7-b0b9-b2a22cd18c2e None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:13.287790 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9461b-6a45-44f7-b0b9-b2a22cd18c2e None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:13.294128 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8196ea-4a34-4cda-b838-7b1fcf98fdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:13.294937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8196ea-4a34-4cda-b838-7b1fcf98fdce admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:13.295324 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8196ea-4a34-4cda-b838-7b1fcf98fdce admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:13.295608 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8196ea-4a34-4cda-b838-7b1fcf98fdce admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:13.296394 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8196ea-4a34-4cda-b838-7b1fcf98fdce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:34:13.296918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8196ea-4a34-4cda-b838-7b1fcf98fdce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:13.298597 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8196ea-4a34-4cda-b838-7b1fcf98fdce admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:13.318965 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 988/1964] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:34:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:34:13.324269 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e11b3-1fde-4de0-904b-b533005feffe None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:13.345579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e11b3-1fde-4de0-904b-b533005feffe None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:13.358200 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973bdb3c-8381-4a5f-bfb8-4aee9c102402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:13.359329 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bdb3c-8381-4a5f-bfb8-4aee9c102402 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:13.359761 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bdb3c-8381-4a5f-bfb8-4aee9c102402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:13.360314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bdb3c-8381-4a5f-bfb8-4aee9c102402 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:13.361781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973bdb3c-8381-4a5f-bfb8-4aee9c102402 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:13.365344 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973bdb3c-8381-4a5f-bfb8-4aee9c102402 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:13.426288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 977/1965] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:34:13 2022] POST /identity/v3/projects => generated 382 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:34:13.433586 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8196ea-4a34-4cda-b838-7b1fcf98fdce admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:13.457239 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8196ea-4a34-4cda-b838-7b1fcf98fdce admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:13.519442 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:13.581488 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b922c-b277-42bd-94e3-77dba317a51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:13.582151 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b922c-b277-42bd-94e3-77dba317a51b admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:13.582892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b922c-b277-42bd-94e3-77dba317a51b admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:13.583170 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b922c-b277-42bd-94e3-77dba317a51b admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:13.583865 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b922c-b277-42bd-94e3-77dba317a51b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:13.585887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b922c-b277-42bd-94e3-77dba317a51b admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:13.612294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 989/1966] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:34:13 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:34:13.615762 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973bdb3c-8381-4a5f-bfb8-4aee9c102402 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:13.627090 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973bdb3c-8381-4a5f-bfb8-4aee9c102402 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:13.632196 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfcdb2f-646f-443f-8c8c-2762783c1f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:13.632858 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfcdb2f-646f-443f-8c8c-2762783c1f24 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:13.633172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfcdb2f-646f-443f-8c8c-2762783c1f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:13.633485 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfcdb2f-646f-443f-8c8c-2762783c1f24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:13.634202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfcdb2f-646f-443f-8c8c-2762783c1f24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:13.636269 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfcdb2f-646f-443f-8c8c-2762783c1f24 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:13.652148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 978/1967] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:34:13 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:34:13.653624 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b922c-b277-42bd-94e3-77dba317a51b admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:13.665505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b922c-b277-42bd-94e3-77dba317a51b admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:13.679589 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53be94-902e-4915-9aa7-9fddabb29107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:13.680322 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53be94-902e-4915-9aa7-9fddabb29107 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:13.680611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53be94-902e-4915-9aa7-9fddabb29107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:13.680891 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53be94-902e-4915-9aa7-9fddabb29107 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:13.681616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53be94-902e-4915-9aa7-9fddabb29107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:13.683319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53be94-902e-4915-9aa7-9fddabb29107 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:13.695020 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 990/1968] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:34:13 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:34:13.699726 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfcdb2f-646f-443f-8c8c-2762783c1f24 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:13.711039 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfcdb2f-646f-443f-8c8c-2762783c1f24 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:13.716139 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096a24d-9c18-4fbd-a114-37fdcc9751b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:13.716839 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096a24d-9c18-4fbd-a114-37fdcc9751b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:13.717129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096a24d-9c18-4fbd-a114-37fdcc9751b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:13.717413 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096a24d-9c18-4fbd-a114-37fdcc9751b3 admin admin] PATH_INFO: `/v3/projects/0f7489a08d43419a86e3c3476426282b/users/3f8047fb9985488483ba817817f60641/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:13.735796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096a24d-9c18-4fbd-a114-37fdcc9751b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f7489a08d43419a86e3c3476426282b, user_id=3f8047fb9985488483ba817817f60641, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:13.739036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096a24d-9c18-4fbd-a114-37fdcc9751b3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:13.749924 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:13.756719 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 979/1969] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:34:13 2022] PUT /identity/v3/projects/0f7489a08d43419a86e3c3476426282b/users/3f8047fb9985488483ba817817f60641/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:13.767594 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9870e52-7587-4d1b-aba1-9bda388ad939 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:13.767889 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9870e52-7587-4d1b-aba1-9bda388ad939 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:13.768411 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9870e52-7587-4d1b-aba1-9bda388ad939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:13.799786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-f9870e52-7587-4d1b-aba1-9bda388ad939 None None] MFA Rules not processed for user `3f8047fb9985488483ba817817f60641`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:13.801268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-f9870e52-7587-4d1b-aba1-9bda388ad939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:13.884646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 991/1970] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:13.889473 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096a24d-9c18-4fbd-a114-37fdcc9751b3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:13.971840 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096a24d-9c18-4fbd-a114-37fdcc9751b3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:13.977109 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d4957-8a6c-4352-9cf3-0fb320760db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:13.978142 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d4957-8a6c-4352-9cf3-0fb320760db7 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:13.978433 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d4957-8a6c-4352-9cf3-0fb320760db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:13.978714 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d4957-8a6c-4352-9cf3-0fb320760db7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:13.979415 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7d4957-8a6c-4352-9cf3-0fb320760db7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:13.981582 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7d4957-8a6c-4352-9cf3-0fb320760db7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:14.015683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 980/1971] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:34:13 2022] POST /identity/v3/projects => generated 384 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:34:14.021122 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9870e52-7587-4d1b-aba1-9bda388ad939 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:14.034265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9870e52-7587-4d1b-aba1-9bda388ad939 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:14.116851 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c32e2-135d-4228-a42d-dcc5fe04aee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:14.117616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c32e2-135d-4228-a42d-dcc5fe04aee1 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:14.117995 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c32e2-135d-4228-a42d-dcc5fe04aee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:14.118347 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c32e2-135d-4228-a42d-dcc5fe04aee1 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:14.119109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8c32e2-135d-4228-a42d-dcc5fe04aee1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:14.121408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8c32e2-135d-4228-a42d-dcc5fe04aee1 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:14.147165 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 992/1972] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:34:14 2022] POST /identity/v3/users => generated 299 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:34:14.152194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d4957-8a6c-4352-9cf3-0fb320760db7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:14.164106 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d4957-8a6c-4352-9cf3-0fb320760db7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:14.170586 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f0e40-d9bd-4f8c-b3ec-d65e15ad7f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:14.171404 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f0e40-d9bd-4f8c-b3ec-d65e15ad7f6f admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:14.171578 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f0e40-d9bd-4f8c-b3ec-d65e15ad7f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:14.171881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f0e40-d9bd-4f8c-b3ec-d65e15ad7f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:14.172705 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f0e40-d9bd-4f8c-b3ec-d65e15ad7f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:14.174360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f0e40-d9bd-4f8c-b3ec-d65e15ad7f6f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:14.187009 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 981/1973] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:34:14 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:34:14.193831 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c32e2-135d-4228-a42d-dcc5fe04aee1 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:14.206092 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c32e2-135d-4228-a42d-dcc5fe04aee1 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:14.211698 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397eb224-9a0c-4ff0-88eb-2003c7387c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:14.212451 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397eb224-9a0c-4ff0-88eb-2003c7387c97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:14.212730 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397eb224-9a0c-4ff0-88eb-2003c7387c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:14.213007 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397eb224-9a0c-4ff0-88eb-2003c7387c97 admin admin] PATH_INFO: `/v3/projects/0032b8af960144ceab5d0d7e3d0cbfd1/users/8fb656dbc8ee4810b4e0ddd5a385e3a2/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:14.230814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397eb224-9a0c-4ff0-88eb-2003c7387c97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0032b8af960144ceab5d0d7e3d0cbfd1, user_id=8fb656dbc8ee4810b4e0ddd5a385e3a2, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:14.233649 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397eb224-9a0c-4ff0-88eb-2003c7387c97 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:14.252033 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 993/1974] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:34:14 2022] PUT /identity/v3/projects/0032b8af960144ceab5d0d7e3d0cbfd1/users/8fb656dbc8ee4810b4e0ddd5a385e3a2/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:14.256657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f0e40-d9bd-4f8c-b3ec-d65e15ad7f6f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:14.268786 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f0e40-d9bd-4f8c-b3ec-d65e15ad7f6f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:14.334309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bb1a7-d12c-4c64-9cfa-9f5b06b72302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:14.334844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bb1a7-d12c-4c64-9cfa-9f5b06b72302 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:14.335128 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bb1a7-d12c-4c64-9cfa-9f5b06b72302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:14.335404 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bb1a7-d12c-4c64-9cfa-9f5b06b72302 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:14.336155 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0bb1a7-d12c-4c64-9cfa-9f5b06b72302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:14.338015 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0bb1a7-d12c-4c64-9cfa-9f5b06b72302 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:14.350143 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 982/1975] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:34:14 2022] GET /identity/v3/roles => generated 1427 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:34:14.355086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397eb224-9a0c-4ff0-88eb-2003c7387c97 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:14.361228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:14.372268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397eb224-9a0c-4ff0-88eb-2003c7387c97 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:14.381325 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa0c0a-b6e4-4154-8c2b-5dd4422d0185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:14.382727 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa0c0a-b6e4-4154-8c2b-5dd4422d0185 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:14.383222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa0c0a-b6e4-4154-8c2b-5dd4422d0185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:14.383734 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa0c0a-b6e4-4154-8c2b-5dd4422d0185 admin admin] PATH_INFO: `/v3/projects/0032b8af960144ceab5d0d7e3d0cbfd1/users/8fb656dbc8ee4810b4e0ddd5a385e3a2/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:14.387669 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa0c0a-b6e4-4154-8c2b-5dd4422d0185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0032b8af960144ceab5d0d7e3d0cbfd1, user_id=8fb656dbc8ee4810b4e0ddd5a385e3a2, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:14.393371 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa0c0a-b6e4-4154-8c2b-5dd4422d0185 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:14.413197 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 994/1976] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:34:14 2022] PUT /identity/v3/projects/0032b8af960144ceab5d0d7e3d0cbfd1/users/8fb656dbc8ee4810b4e0ddd5a385e3a2/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:14.425634 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68c0d4-b873-49f6-8057-6deff3ff4456 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:14.425920 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68c0d4-b873-49f6-8057-6deff3ff4456 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:14.426202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68c0d4-b873-49f6-8057-6deff3ff4456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:14.459121 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-1c68c0d4-b873-49f6-8057-6deff3ff4456 None None] MFA Rules not processed for user `8fb656dbc8ee4810b4e0ddd5a385e3a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:14.460552 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1c68c0d4-b873-49f6-8057-6deff3ff4456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:14.531449 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:14.542486 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 983/1977] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:14 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:14.559576 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa0c0a-b6e4-4154-8c2b-5dd4422d0185 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:14.561778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-43fa0c0a-b6e4-4154-8c2b-5dd4422d0185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:14.639082 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa0c0a-b6e4-4154-8c2b-5dd4422d0185 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:14.643732 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede24fc-758c-4ebe-bd18-1030858f957e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:14.644454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede24fc-758c-4ebe-bd18-1030858f957e service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:14.644739 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede24fc-758c-4ebe-bd18-1030858f957e service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:14.645012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede24fc-758c-4ebe-bd18-1030858f957e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:14.658534 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede24fc-758c-4ebe-bd18-1030858f957e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:14.660096 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede24fc-758c-4ebe-bd18-1030858f957e service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:14.672796 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 995/1978] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:34:14 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:14.798876 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68c0d4-b873-49f6-8057-6deff3ff4456 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:14.810646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68c0d4-b873-49f6-8057-6deff3ff4456 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:14.816145 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52ef66-58bb-4741-8738-4422698f20d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:14.817674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52ef66-58bb-4741-8738-4422698f20d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:14.817963 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52ef66-58bb-4741-8738-4422698f20d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:14.818277 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52ef66-58bb-4741-8738-4422698f20d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:14.887077 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d52ef66-58bb-4741-8738-4422698f20d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:14.888587 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d52ef66-58bb-4741-8738-4422698f20d0 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:14.899319 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 984/1979] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:34:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:22.936300 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede24fc-758c-4ebe-bd18-1030858f957e service cinder] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:23.052469 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede24fc-758c-4ebe-bd18-1030858f957e service cinder] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:23.060142 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f64d73-b487-4144-8177-933a7aca2e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:23.060848 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f64d73-b487-4144-8177-933a7aca2e9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:23.061140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f64d73-b487-4144-8177-933a7aca2e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:23.061416 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f64d73-b487-4144-8177-933a7aca2e9d admin admin] PATH_INFO: `/v3/users/3f8047fb9985488483ba817817f60641` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:23.062829 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f64d73-b487-4144-8177-933a7aca2e9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f8047fb9985488483ba817817f60641)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:23.064927 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f64d73-b487-4144-8177-933a7aca2e9d admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:23.140644 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-93f64d73-b487-4144-8177-933a7aca2e9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f8047fb9985488483ba817817f60641'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:23.141728 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-93f64d73-b487-4144-8177-933a7aca2e9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f8047fb9985488483ba817817f60641'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:23.169901 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-93f64d73-b487-4144-8177-933a7aca2e9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f8047fb9985488483ba817817f60641'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:23.190738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-93f64d73-b487-4144-8177-933a7aca2e9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f8047fb9985488483ba817817f60641'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:23.208770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-93f64d73-b487-4144-8177-933a7aca2e9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f8047fb9985488483ba817817f60641'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:23.218254 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 996/1980] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:34:22 2022] DELETE /identity/v3/users/3f8047fb9985488483ba817817f60641 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:23.225527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab390df7-74e6-4ccc-8c22-80dc33181b96 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:23.226436 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab390df7-74e6-4ccc-8c22-80dc33181b96 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:23.226436 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab390df7-74e6-4ccc-8c22-80dc33181b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:23.256765 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ab390df7-74e6-4ccc-8c22-80dc33181b96 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:23.258854 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ab390df7-74e6-4ccc-8c22-80dc33181b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:23.317258 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f64d73-b487-4144-8177-933a7aca2e9d admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:23.319229 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-93f64d73-b487-4144-8177-933a7aca2e9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:23.340645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:23.358061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 985/1981] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:23.372872 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab390df7-74e6-4ccc-8c22-80dc33181b96 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:23.372872 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ab390df7-74e6-4ccc-8c22-80dc33181b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:23.434820 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f64d73-b487-4144-8177-933a7aca2e9d admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:23.446650 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0fe45-6e45-4d06-91c7-0699ef7ce3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:23.448067 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0fe45-6e45-4d06-91c7-0699ef7ce3fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:23.449230 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0fe45-6e45-4d06-91c7-0699ef7ce3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:23.449561 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab390df7-74e6-4ccc-8c22-80dc33181b96 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:23.449750 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0fe45-6e45-4d06-91c7-0699ef7ce3fb admin admin] PATH_INFO: `/v3/users/acea7f13135346ea9b5ce1d06abb4491` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:23.452362 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb0fe45-6e45-4d06-91c7-0699ef7ce3fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=acea7f13135346ea9b5ce1d06abb4491)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:23.453543 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4b6c4-e559-49e0-bc1c-9961d153f419 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:23.454207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4b6c4-e559-49e0-bc1c-9961d153f419 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:23.454476 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4b6c4-e559-49e0-bc1c-9961d153f419 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:23.454753 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4b6c4-e559-49e0-bc1c-9961d153f419 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:23.465950 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4b6c4-e559-49e0-bc1c-9961d153f419 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:23.467675 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4b6c4-e559-49e0-bc1c-9961d153f419 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:23.472551 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb0fe45-6e45-4d06-91c7-0699ef7ce3fb admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:23.482587 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 986/1982] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:34:23 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:34:23.566827 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-bbb0fe45-6e45-4d06-91c7-0699ef7ce3fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acea7f13135346ea9b5ce1d06abb4491'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:23.568744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-bbb0fe45-6e45-4d06-91c7-0699ef7ce3fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acea7f13135346ea9b5ce1d06abb4491'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:23.614639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-bbb0fe45-6e45-4d06-91c7-0699ef7ce3fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acea7f13135346ea9b5ce1d06abb4491'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:23.626915 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-bbb0fe45-6e45-4d06-91c7-0699ef7ce3fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acea7f13135346ea9b5ce1d06abb4491'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:23.642005 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-bbb0fe45-6e45-4d06-91c7-0699ef7ce3fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acea7f13135346ea9b5ce1d06abb4491'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:23.658623 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 997/1983] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:34:23 2022] DELETE /identity/v3/users/acea7f13135346ea9b5ce1d06abb4491 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:34:23.667181 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babf166-0d58-4c84-9327-35864123b5a3 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:23.667181 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babf166-0d58-4c84-9327-35864123b5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:23.667181 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babf166-0d58-4c84-9327-35864123b5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:23.712713 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4babf166-0d58-4c84-9327-35864123b5a3 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:23.715341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4babf166-0d58-4c84-9327-35864123b5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:23.816835 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:23.870290 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 987/1984] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:23.891353 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0fe45-6e45-4d06-91c7-0699ef7ce3fb admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:23.894385 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-bbb0fe45-6e45-4d06-91c7-0699ef7ce3fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:23.973046 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4babf166-0d58-4c84-9327-35864123b5a3 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:23.975447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4babf166-0d58-4c84-9327-35864123b5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:24.000485 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4babf166-0d58-4c84-9327-35864123b5a3 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:24.010522 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6921b-fa1b-41e6-9bc8-303d92e9cbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:24.011894 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6921b-fa1b-41e6-9bc8-303d92e9cbe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:24.012337 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6921b-fa1b-41e6-9bc8-303d92e9cbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:24.012738 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6921b-fa1b-41e6-9bc8-303d92e9cbe5 admin admin] PATH_INFO: `/v3/projects/0f7489a08d43419a86e3c3476426282b` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:24.014574 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6921b-fa1b-41e6-9bc8-303d92e9cbe5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f7489a08d43419a86e3c3476426282b)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:24.017759 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6921b-fa1b-41e6-9bc8-303d92e9cbe5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:24.051563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0fe45-6e45-4d06-91c7-0699ef7ce3fb admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:24.060316 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9b4db-d8f4-42c5-9aa7-d4e9d6926c42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:24.061185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9b4db-d8f4-42c5-9aa7-d4e9d6926c42 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:24.061393 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9b4db-d8f4-42c5-9aa7-d4e9d6926c42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:24.061659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9b4db-d8f4-42c5-9aa7-d4e9d6926c42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:24.076321 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d9b4db-d8f4-42c5-9aa7-d4e9d6926c42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:24.077846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d9b4db-d8f4-42c5-9aa7-d4e9d6926c42 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:24.093483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 998/1985] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:34:23 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:24.178824 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3ae6921b-fa1b-41e6-9bc8-303d92e9cbe5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f7489a08d43419a86e3c3476426282b', 'request_id': 'req-3ae6921b-fa1b-41e6-9bc8-303d92e9cbe5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:24.188362 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3ae6921b-fa1b-41e6-9bc8-303d92e9cbe5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f7489a08d43419a86e3c3476426282b', 'request_id': 'req-3ae6921b-fa1b-41e6-9bc8-303d92e9cbe5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:24.198590 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3ae6921b-fa1b-41e6-9bc8-303d92e9cbe5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f7489a08d43419a86e3c3476426282b', 'request_id': 'req-3ae6921b-fa1b-41e6-9bc8-303d92e9cbe5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:24.233606 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3ae6921b-fa1b-41e6-9bc8-303d92e9cbe5 admin admin] The token cache is being invalidate because project 0f7489a08d43419a86e3c3476426282b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:34:24.234287 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3ae6921b-fa1b-41e6-9bc8-303d92e9cbe5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:24.234883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3ae6921b-fa1b-41e6-9bc8-303d92e9cbe5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:24.237685 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 988/1986] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:34:23 2022] DELETE /identity/v3/projects/0f7489a08d43419a86e3c3476426282b => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:24.243202 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9b4db-d8f4-42c5-9aa7-d4e9d6926c42 service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:24.263379 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9b4db-d8f4-42c5-9aa7-d4e9d6926c42 service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:24.315265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:24.370373 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845daac1-e4c1-4fab-8d8a-79b382ae149b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:24.372303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845daac1-e4c1-4fab-8d8a-79b382ae149b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:24.372303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845daac1-e4c1-4fab-8d8a-79b382ae149b admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:24.372303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845daac1-e4c1-4fab-8d8a-79b382ae149b admin admin] PATH_INFO: `/v3/users/8fb656dbc8ee4810b4e0ddd5a385e3a2` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:24.373033 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845daac1-e4c1-4fab-8d8a-79b382ae149b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fb656dbc8ee4810b4e0ddd5a385e3a2)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:24.375288 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845daac1-e4c1-4fab-8d8a-79b382ae149b admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:24.474840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-845daac1-e4c1-4fab-8d8a-79b382ae149b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fb656dbc8ee4810b4e0ddd5a385e3a2'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:24.478252 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-845daac1-e4c1-4fab-8d8a-79b382ae149b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fb656dbc8ee4810b4e0ddd5a385e3a2'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:24.502893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-845daac1-e4c1-4fab-8d8a-79b382ae149b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fb656dbc8ee4810b4e0ddd5a385e3a2'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:24.547028 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-845daac1-e4c1-4fab-8d8a-79b382ae149b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fb656dbc8ee4810b4e0ddd5a385e3a2'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:24.562773 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6921b-fa1b-41e6-9bc8-303d92e9cbe5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:24.566421 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3ae6921b-fa1b-41e6-9bc8-303d92e9cbe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:24.582770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-845daac1-e4c1-4fab-8d8a-79b382ae149b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fb656dbc8ee4810b4e0ddd5a385e3a2'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:24.595169 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 999/1987] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:34:24 2022] DELETE /identity/v3/users/8fb656dbc8ee4810b4e0ddd5a385e3a2 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:34:24.630115 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6921b-fa1b-41e6-9bc8-303d92e9cbe5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:24.761577 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e5e79-16e3-4856-af61-14a4a98fdf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:24.762912 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e5e79-16e3-4856-af61-14a4a98fdf32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:24.763211 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e5e79-16e3-4856-af61-14a4a98fdf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:24.763543 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e5e79-16e3-4856-af61-14a4a98fdf32 admin admin] PATH_INFO: `/v3/projects/f9c972cb2ce14cb5bd9b9e29d3000b2a` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:24.765238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3e5e79-16e3-4856-af61-14a4a98fdf32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9c972cb2ce14cb5bd9b9e29d3000b2a)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:24.769527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3e5e79-16e3-4856-af61-14a4a98fdf32 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:24.894396 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7a3e5e79-16e3-4856-af61-14a4a98fdf32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9c972cb2ce14cb5bd9b9e29d3000b2a', 'request_id': 'req-7a3e5e79-16e3-4856-af61-14a4a98fdf32'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:24.917922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7a3e5e79-16e3-4856-af61-14a4a98fdf32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9c972cb2ce14cb5bd9b9e29d3000b2a', 'request_id': 'req-7a3e5e79-16e3-4856-af61-14a4a98fdf32'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:24.933806 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7a3e5e79-16e3-4856-af61-14a4a98fdf32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9c972cb2ce14cb5bd9b9e29d3000b2a', 'request_id': 'req-7a3e5e79-16e3-4856-af61-14a4a98fdf32'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:24.934279 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845daac1-e4c1-4fab-8d8a-79b382ae149b admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:24.939093 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:24.943678 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-845daac1-e4c1-4fab-8d8a-79b382ae149b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:24.978385 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845daac1-e4c1-4fab-8d8a-79b382ae149b admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:24.980476 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7a3e5e79-16e3-4856-af61-14a4a98fdf32 admin admin] The token cache is being invalidate because project f9c972cb2ce14cb5bd9b9e29d3000b2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:34:24.983226 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7a3e5e79-16e3-4856-af61-14a4a98fdf32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:24.985320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7a3e5e79-16e3-4856-af61-14a4a98fdf32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:24.988653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 989/1988] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:34:24 2022] DELETE /identity/v3/projects/f9c972cb2ce14cb5bd9b9e29d3000b2a => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:24.999495 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e5e79-16e3-4856-af61-14a4a98fdf32 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:25.005665 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7a3e5e79-16e3-4856-af61-14a4a98fdf32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:25.129610 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae06e1d-7f9e-408e-a9be-28a36897f5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:25.130375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae06e1d-7f9e-408e-a9be-28a36897f5f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:25.130673 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae06e1d-7f9e-408e-a9be-28a36897f5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:25.131020 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae06e1d-7f9e-408e-a9be-28a36897f5f0 admin admin] PATH_INFO: `/v3/users/9ea8d0d05b2a41429ccc77cc291a31e6` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:25.135552 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae06e1d-7f9e-408e-a9be-28a36897f5f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ea8d0d05b2a41429ccc77cc291a31e6)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:25.137735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae06e1d-7f9e-408e-a9be-28a36897f5f0 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:25.230327 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e5e79-16e3-4856-af61-14a4a98fdf32 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:25.244841 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b553c48-9059-49b8-92e3-3b2e1b415ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:25.245934 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0ae06e1d-7f9e-408e-a9be-28a36897f5f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ea8d0d05b2a41429ccc77cc291a31e6'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:25.246526 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b553c48-9059-49b8-92e3-3b2e1b415ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:25.246811 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b553c48-9059-49b8-92e3-3b2e1b415ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:25.247116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b553c48-9059-49b8-92e3-3b2e1b415ea6 admin admin] PATH_INFO: `/v3/users/31b4182e6beb438e8e881e699ef2ef59` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:25.248107 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0ae06e1d-7f9e-408e-a9be-28a36897f5f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ea8d0d05b2a41429ccc77cc291a31e6'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:25.250467 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b553c48-9059-49b8-92e3-3b2e1b415ea6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31b4182e6beb438e8e881e699ef2ef59)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:25.254396 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b553c48-9059-49b8-92e3-3b2e1b415ea6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:25.262618 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:25.286739 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0ae06e1d-7f9e-408e-a9be-28a36897f5f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ea8d0d05b2a41429ccc77cc291a31e6'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:25.303910 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0ae06e1d-7f9e-408e-a9be-28a36897f5f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ea8d0d05b2a41429ccc77cc291a31e6'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:25.316785 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0ae06e1d-7f9e-408e-a9be-28a36897f5f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ea8d0d05b2a41429ccc77cc291a31e6'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:25.328249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1000/1989] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:34:24 2022] DELETE /identity/v3/users/9ea8d0d05b2a41429ccc77cc291a31e6 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:25.329403 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae06e1d-7f9e-408e-a9be-28a36897f5f0 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:25.331052 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-0ae06e1d-7f9e-408e-a9be-28a36897f5f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:25.371892 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4b553c48-9059-49b8-92e3-3b2e1b415ea6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31b4182e6beb438e8e881e699ef2ef59'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:25.392399 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4b553c48-9059-49b8-92e3-3b2e1b415ea6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31b4182e6beb438e8e881e699ef2ef59'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:25.393973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4b553c48-9059-49b8-92e3-3b2e1b415ea6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31b4182e6beb438e8e881e699ef2ef59'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:25.420257 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae06e1d-7f9e-408e-a9be-28a36897f5f0 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:25.426269 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5744b27-b4ad-4517-9b25-67948dc74c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:25.427057 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5744b27-b4ad-4517-9b25-67948dc74c06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:25.427345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5744b27-b4ad-4517-9b25-67948dc74c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:25.427650 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5744b27-b4ad-4517-9b25-67948dc74c06 admin admin] PATH_INFO: `/v3/projects/0032b8af960144ceab5d0d7e3d0cbfd1` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:25.429712 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5744b27-b4ad-4517-9b25-67948dc74c06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0032b8af960144ceab5d0d7e3d0cbfd1)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:25.429864 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4b553c48-9059-49b8-92e3-3b2e1b415ea6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31b4182e6beb438e8e881e699ef2ef59'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:25.430568 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4b553c48-9059-49b8-92e3-3b2e1b415ea6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31b4182e6beb438e8e881e699ef2ef59'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:25.431746 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5744b27-b4ad-4517-9b25-67948dc74c06 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:25.454101 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 990/1990] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:34:24 2022] DELETE /identity/v3/users/31b4182e6beb438e8e881e699ef2ef59 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:25.468249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b4295-f45d-401d-931e-335134cce2d4 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:25.468594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b4295-f45d-401d-931e-335134cce2d4 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:25.470567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b4295-f45d-401d-931e-335134cce2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:25.507044 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f48b4295-f45d-401d-931e-335134cce2d4 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:25.508601 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f48b4295-f45d-401d-931e-335134cce2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:25.661219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a5744b27-b4ad-4517-9b25-67948dc74c06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0032b8af960144ceab5d0d7e3d0cbfd1', 'request_id': 'req-a5744b27-b4ad-4517-9b25-67948dc74c06'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:25.677970 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a5744b27-b4ad-4517-9b25-67948dc74c06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0032b8af960144ceab5d0d7e3d0cbfd1', 'request_id': 'req-a5744b27-b4ad-4517-9b25-67948dc74c06'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:25.692637 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a5744b27-b4ad-4517-9b25-67948dc74c06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0032b8af960144ceab5d0d7e3d0cbfd1', 'request_id': 'req-a5744b27-b4ad-4517-9b25-67948dc74c06'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:25.722994 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a5744b27-b4ad-4517-9b25-67948dc74c06 admin admin] The token cache is being invalidate because project 0032b8af960144ceab5d0d7e3d0cbfd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:34:25.723733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a5744b27-b4ad-4517-9b25-67948dc74c06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:25.724727 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a5744b27-b4ad-4517-9b25-67948dc74c06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:25.726942 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1001/1991] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:34:25 2022] DELETE /identity/v3/projects/0032b8af960144ceab5d0d7e3d0cbfd1 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:25.736694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e103d-6d30-4aea-9ca5-d2963c6ece9c None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:25.737053 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e103d-6d30-4aea-9ca5-d2963c6ece9c None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:25.737257 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e103d-6d30-4aea-9ca5-d2963c6ece9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:25.741580 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 991/1992] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:25.767071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b4295-f45d-401d-931e-335134cce2d4 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:25.771893 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f48b4295-f45d-401d-931e-335134cce2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:25.776480 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-0c2e103d-6d30-4aea-9ca5-d2963c6ece9c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:25.778005 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-0c2e103d-6d30-4aea-9ca5-d2963c6ece9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:25.869316 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:25.914983 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1002/1993] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:34:25.927319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7895554-213b-4eeb-889a-117bed056f3e None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:25.927712 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7895554-213b-4eeb-889a-117bed056f3e None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:25.928098 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7895554-213b-4eeb-889a-117bed056f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:25.943317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:25.980872 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-f7895554-213b-4eeb-889a-117bed056f3e None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:25.980872 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-f7895554-213b-4eeb-889a-117bed056f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:25.989928 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1003/1994] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:34:25.994675 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7895554-213b-4eeb-889a-117bed056f3e None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:26.019215 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7895554-213b-4eeb-889a-117bed056f3e None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:26.025823 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b4295-f45d-401d-931e-335134cce2d4 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:26.030340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcf1603-ce82-4f01-8a44-ac77ef023c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:26.033245 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf1603-ce82-4f01-8a44-ac77ef023c03 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:26.034398 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf1603-ce82-4f01-8a44-ac77ef023c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:26.035635 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf1603-ce82-4f01-8a44-ac77ef023c03 admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:26.037439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcf1603-ce82-4f01-8a44-ac77ef023c03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:34:26.038353 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcf1603-ce82-4f01-8a44-ac77ef023c03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:26.042131 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcf1603-ce82-4f01-8a44-ac77ef023c03 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:26.079732 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1004/1995] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:34:25 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:34:26.087066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcf1603-ce82-4f01-8a44-ac77ef023c03 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:26.105172 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcf1603-ce82-4f01-8a44-ac77ef023c03 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:26.121276 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff6e9d-c8d0-4610-85a4-18910aab9155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:26.122356 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff6e9d-c8d0-4610-85a4-18910aab9155 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:26.122818 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff6e9d-c8d0-4610-85a4-18910aab9155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:26.123304 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff6e9d-c8d0-4610-85a4-18910aab9155 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:26.125031 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff6e9d-c8d0-4610-85a4-18910aab9155 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:26.128908 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff6e9d-c8d0-4610-85a4-18910aab9155 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:26.140842 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e5535-e953-4215-b0d9-cb294ba69daa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:26.140842 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e5535-e953-4215-b0d9-cb294ba69daa service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:26.140842 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e5535-e953-4215-b0d9-cb294ba69daa service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:26.140842 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e5535-e953-4215-b0d9-cb294ba69daa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:26.154564 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e5535-e953-4215-b0d9-cb294ba69daa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:26.156867 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e5535-e953-4215-b0d9-cb294ba69daa service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:26.186350 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 992/1996] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:34:25 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:26.192224 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1005/1997] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:34:26 2022] POST /identity/v3/projects => generated 408 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:34:26.196546 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e5535-e953-4215-b0d9-cb294ba69daa service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:26.207758 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e5535-e953-4215-b0d9-cb294ba69daa service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:26.363925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ead9a-9446-4450-a4d7-fceb0ad8ca71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:26.365171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ead9a-9446-4450-a4d7-fceb0ad8ca71 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:26.365492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ead9a-9446-4450-a4d7-fceb0ad8ca71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:26.365971 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ead9a-9446-4450-a4d7-fceb0ad8ca71 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:26.367180 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ead9a-9446-4450-a4d7-fceb0ad8ca71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:26.371045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ead9a-9446-4450-a4d7-fceb0ad8ca71 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:26.411769 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 993/1998] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:34:26 2022] POST /identity/v3/users => generated 305 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:34:26.417173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff6e9d-c8d0-4610-85a4-18910aab9155 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:26.446738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff6e9d-c8d0-4610-85a4-18910aab9155 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:26.459673 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b0534-d379-4477-b31a-7d065533ff74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:26.460717 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b0534-d379-4477-b31a-7d065533ff74 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:26.461003 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b0534-d379-4477-b31a-7d065533ff74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:26.461537 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b0534-d379-4477-b31a-7d065533ff74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:26.462893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b0534-d379-4477-b31a-7d065533ff74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:26.465437 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b0534-d379-4477-b31a-7d065533ff74 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:26.495863 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ead9a-9446-4450-a4d7-fceb0ad8ca71 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:26.500096 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1006/1999] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:34:26 2022] GET /identity/v3/roles => generated 1427 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:34:26.507589 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ead9a-9446-4450-a4d7-fceb0ad8ca71 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:26.508825 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b0534-d379-4477-b31a-7d065533ff74 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:26.512196 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-659b0534-d379-4477-b31a-7d065533ff74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:26.514569 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd1018-a1bb-49fa-b4c5-878d00f5e47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:26.515411 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd1018-a1bb-49fa-b4c5-878d00f5e47b admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:26.515896 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd1018-a1bb-49fa-b4c5-878d00f5e47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:26.516354 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd1018-a1bb-49fa-b4c5-878d00f5e47b admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:26.517238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd1018-a1bb-49fa-b4c5-878d00f5e47b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:26.519910 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd1018-a1bb-49fa-b4c5-878d00f5e47b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:26.537589 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b0534-d379-4477-b31a-7d065533ff74 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:26.543327 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad02db-7230-4844-90b0-2fffe548262a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:26.545229 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad02db-7230-4844-90b0-2fffe548262a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:26.545526 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad02db-7230-4844-90b0-2fffe548262a admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:26.545800 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad02db-7230-4844-90b0-2fffe548262a admin admin] PATH_INFO: `/v3/projects/8f50765ec5854049bafcb12b33d24593` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:26.548248 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad02db-7230-4844-90b0-2fffe548262a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f50765ec5854049bafcb12b33d24593)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:26.550114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 994/2000] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:34:26 2022] GET /identity/v3/roles => generated 1427 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:34:26.551336 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad02db-7230-4844-90b0-2fffe548262a admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:26.556643 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd1018-a1bb-49fa-b4c5-878d00f5e47b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:26.590679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd1018-a1bb-49fa-b4c5-878d00f5e47b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:26.606986 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f0104-86af-4621-9da3-02d08eaf5342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:26.608802 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f0104-86af-4621-9da3-02d08eaf5342 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:26.609110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f0104-86af-4621-9da3-02d08eaf5342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:26.609986 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f0104-86af-4621-9da3-02d08eaf5342 admin admin] PATH_INFO: `/v3/projects/dd26e858de04440fada96f1845a16cf4/users/ee37026a989a40bebb2dd6bfeee4193d/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:26.637320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f0104-86af-4621-9da3-02d08eaf5342 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd26e858de04440fada96f1845a16cf4, user_id=ee37026a989a40bebb2dd6bfeee4193d, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:26.640482 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f0104-86af-4621-9da3-02d08eaf5342 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:26.644526 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-08ad02db-7230-4844-90b0-2fffe548262a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f50765ec5854049bafcb12b33d24593', 'request_id': 'req-08ad02db-7230-4844-90b0-2fffe548262a'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:26.665106 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-08ad02db-7230-4844-90b0-2fffe548262a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f50765ec5854049bafcb12b33d24593', 'request_id': 'req-08ad02db-7230-4844-90b0-2fffe548262a'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:26.669804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 995/2001] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:34:26 2022] PUT /identity/v3/projects/dd26e858de04440fada96f1845a16cf4/users/ee37026a989a40bebb2dd6bfeee4193d/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:26.679492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f0104-86af-4621-9da3-02d08eaf5342 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:26.681048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5e3f0104-86af-4621-9da3-02d08eaf5342 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:26.692341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-08ad02db-7230-4844-90b0-2fffe548262a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f50765ec5854049bafcb12b33d24593', 'request_id': 'req-08ad02db-7230-4844-90b0-2fffe548262a'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:26.709944 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-08ad02db-7230-4844-90b0-2fffe548262a admin admin] The token cache is being invalidate because project 8f50765ec5854049bafcb12b33d24593 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:34:26.710500 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-08ad02db-7230-4844-90b0-2fffe548262a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:26.712345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-08ad02db-7230-4844-90b0-2fffe548262a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:26.714091 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1007/2002] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:34:26 2022] DELETE /identity/v3/projects/8f50765ec5854049bafcb12b33d24593 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:26.715617 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c91e7d-93f1-4ff3-ab6c-935528efbdb2 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:26.715912 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c91e7d-93f1-4ff3-ab6c-935528efbdb2 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:26.716263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c91e7d-93f1-4ff3-ab6c-935528efbdb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:26.739910 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:26.757077 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-61c91e7d-93f1-4ff3-ab6c-935528efbdb2 None None] MFA Rules not processed for user `ee37026a989a40bebb2dd6bfeee4193d`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:26.758722 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-61c91e7d-93f1-4ff3-ab6c-935528efbdb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:26.801476 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f0104-86af-4621-9da3-02d08eaf5342 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:26.809098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998b099-d870-4495-8ff9-aede28d5ede6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:26.812000 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998b099-d870-4495-8ff9-aede28d5ede6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:26.813319 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998b099-d870-4495-8ff9-aede28d5ede6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:26.813848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998b099-d870-4495-8ff9-aede28d5ede6 admin admin] PATH_INFO: `/v3/projects/df56de1ea1be4affa107cded85597809` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:26.816463 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5998b099-d870-4495-8ff9-aede28d5ede6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df56de1ea1be4affa107cded85597809)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:26.818664 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5998b099-d870-4495-8ff9-aede28d5ede6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:26.876666 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1008/2003] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:26 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:26.879127 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bdf69-70f8-4a42-8ea1-c1232b1573f7 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:26.879127 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bdf69-70f8-4a42-8ea1-c1232b1573f7 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:26.879127 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bdf69-70f8-4a42-8ea1-c1232b1573f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:26.894475 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5998b099-d870-4495-8ff9-aede28d5ede6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df56de1ea1be4affa107cded85597809', 'request_id': 'req-5998b099-d870-4495-8ff9-aede28d5ede6'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:26.913370 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5998b099-d870-4495-8ff9-aede28d5ede6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df56de1ea1be4affa107cded85597809', 'request_id': 'req-5998b099-d870-4495-8ff9-aede28d5ede6'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:26.921979 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-9c7bdf69-70f8-4a42-8ea1-c1232b1573f7 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:26.922754 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5998b099-d870-4495-8ff9-aede28d5ede6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df56de1ea1be4affa107cded85597809', 'request_id': 'req-5998b099-d870-4495-8ff9-aede28d5ede6'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:26.923148 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-9c7bdf69-70f8-4a42-8ea1-c1232b1573f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:26.942963 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5998b099-d870-4495-8ff9-aede28d5ede6 admin admin] The token cache is being invalidate because project df56de1ea1be4affa107cded85597809 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:34:26.943326 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5998b099-d870-4495-8ff9-aede28d5ede6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:26.944425 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5998b099-d870-4495-8ff9-aede28d5ede6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:26.949210 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 996/2004] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:34:26 2022] DELETE /identity/v3/projects/df56de1ea1be4affa107cded85597809 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:26.950094 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:26.950264 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998b099-d870-4495-8ff9-aede28d5ede6 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:26.951440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5998b099-d870-4495-8ff9-aede28d5ede6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:27.015113 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1009/2005] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:26 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:27.024588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8eebc-ba83-4e8e-b6fe-e62a1795a7c3 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:27.028458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8eebc-ba83-4e8e-b6fe-e62a1795a7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:27.028746 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8eebc-ba83-4e8e-b6fe-e62a1795a7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:27.049847 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998b099-d870-4495-8ff9-aede28d5ede6 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:27.057327 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-57d8eebc-ba83-4e8e-b6fe-e62a1795a7c3 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:27.058517 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3bfec-ffd4-44b2-82d1-559a1c51627c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:27.059680 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3bfec-ffd4-44b2-82d1-559a1c51627c admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:27.060008 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3bfec-ffd4-44b2-82d1-559a1c51627c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:27.060296 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3bfec-ffd4-44b2-82d1-559a1c51627c admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:27.060715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-57d8eebc-ba83-4e8e-b6fe-e62a1795a7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:27.061091 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d3bfec-ffd4-44b2-82d1-559a1c51627c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:27.063189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d3bfec-ffd4-44b2-82d1-559a1c51627c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:27.072765 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1010/2006] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:27 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:27.075163 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2222dea-53bc-42b2-a7a7-925041d60edb None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:27.075680 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2222dea-53bc-42b2-a7a7-925041d60edb None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:27.076254 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2222dea-53bc-42b2-a7a7-925041d60edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:27.100670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-a2222dea-53bc-42b2-a7a7-925041d60edb None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:27.102074 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-a2222dea-53bc-42b2-a7a7-925041d60edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:27.119638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1011/2007] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:27 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:27.119638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d20cf3-34bd-4702-b8e1-16eca3ef58cc None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:27.119638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d20cf3-34bd-4702-b8e1-16eca3ef58cc None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:27.119638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d20cf3-34bd-4702-b8e1-16eca3ef58cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:27.123311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 997/2008] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:34:26 2022] POST /identity/v3/projects => generated 408 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:34:27.124460 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3bfec-ffd4-44b2-82d1-559a1c51627c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:27.143277 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-34d20cf3-34bd-4702-b8e1-16eca3ef58cc None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:27.145004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-34d20cf3-34bd-4702-b8e1-16eca3ef58cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:27.188073 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:27.219650 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3bfec-ffd4-44b2-82d1-559a1c51627c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:27.220028 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1012/2009] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:27 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:27.220690 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d20cf3-34bd-4702-b8e1-16eca3ef58cc None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:27.234715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4883db-5ef7-48c0-ab89-71794cca73de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:27.236849 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4883db-5ef7-48c0-ab89-71794cca73de admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:27.237445 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4883db-5ef7-48c0-ab89-71794cca73de admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:27.238025 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4883db-5ef7-48c0-ab89-71794cca73de admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:27.238567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d20cf3-34bd-4702-b8e1-16eca3ef58cc None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:27.239536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4883db-5ef7-48c0-ab89-71794cca73de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:34:27.240274 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4883db-5ef7-48c0-ab89-71794cca73de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:27.246358 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b47c68-87f6-4560-a8b5-2a75cef46aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:27.246607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4883db-5ef7-48c0-ab89-71794cca73de admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:27.246694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b47c68-87f6-4560-a8b5-2a75cef46aae admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:27.246694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b47c68-87f6-4560-a8b5-2a75cef46aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:27.246694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b47c68-87f6-4560-a8b5-2a75cef46aae admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:27.246694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b47c68-87f6-4560-a8b5-2a75cef46aae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:27.248275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b47c68-87f6-4560-a8b5-2a75cef46aae admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:27.268752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 998/2010] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:34:27 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:34:27.270427 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4883db-5ef7-48c0-ab89-71794cca73de admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:27.289179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4883db-5ef7-48c0-ab89-71794cca73de admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:27.301042 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bbeb1-5b61-4237-abd9-2767b8ce1688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:27.302354 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bbeb1-5b61-4237-abd9-2767b8ce1688 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:27.303247 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bbeb1-5b61-4237-abd9-2767b8ce1688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:27.303808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bbeb1-5b61-4237-abd9-2767b8ce1688 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:27.305207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8bbeb1-5b61-4237-abd9-2767b8ce1688 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:34:27.305777 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8bbeb1-5b61-4237-abd9-2767b8ce1688 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:27.309087 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8bbeb1-5b61-4237-abd9-2767b8ce1688 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:27.312643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1013/2011] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:34:27 2022] POST /identity/v3/users => generated 311 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:34:27.314554 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b47c68-87f6-4560-a8b5-2a75cef46aae admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:27.329683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b47c68-87f6-4560-a8b5-2a75cef46aae admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:27.335645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb4052-2dc6-4c32-9e8f-6547bd954cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:27.336351 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 999/2012] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:34:27 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:34:27.337562 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bbeb1-5b61-4237-abd9-2767b8ce1688 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:27.349144 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb4052-2dc6-4c32-9e8f-6547bd954cdc admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:27.350494 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb4052-2dc6-4c32-9e8f-6547bd954cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:27.351182 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb4052-2dc6-4c32-9e8f-6547bd954cdc admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:27.352538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cb4052-2dc6-4c32-9e8f-6547bd954cdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:27.355082 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cb4052-2dc6-4c32-9e8f-6547bd954cdc admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:27.359792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bbeb1-5b61-4237-abd9-2767b8ce1688 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:27.368818 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0ec33-2a9f-4b07-8fa3-18890a6e7451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:27.369997 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0ec33-2a9f-4b07-8fa3-18890a6e7451 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:27.370480 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0ec33-2a9f-4b07-8fa3-18890a6e7451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:27.370946 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0ec33-2a9f-4b07-8fa3-18890a6e7451 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:27.372701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0ec33-2a9f-4b07-8fa3-18890a6e7451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:27.375606 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0ec33-2a9f-4b07-8fa3-18890a6e7451 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:27.399682 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1000/2013] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:34:27 2022] GET /identity/v3/roles => generated 1427 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:34:27.401424 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0ec33-2a9f-4b07-8fa3-18890a6e7451 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:27.416092 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1014/2014] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:34:27 2022] POST /identity/v3/projects => generated 370 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:34:27.417295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb4052-2dc6-4c32-9e8f-6547bd954cdc admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:27.419245 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-06cb4052-2dc6-4c32-9e8f-6547bd954cdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:27.422497 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0ec33-2a9f-4b07-8fa3-18890a6e7451 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:27.520962 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb4052-2dc6-4c32-9e8f-6547bd954cdc admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:27.530431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd695a3-7145-45fe-b580-00a6ec4cc600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:27.531817 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd695a3-7145-45fe-b580-00a6ec4cc600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:27.532311 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd695a3-7145-45fe-b580-00a6ec4cc600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:27.532773 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd695a3-7145-45fe-b580-00a6ec4cc600 admin admin] PATH_INFO: `/v3/users/e204358f8e714395b14b51419eadc071` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:27.535135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd695a3-7145-45fe-b580-00a6ec4cc600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e204358f8e714395b14b51419eadc071)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:27.538976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd695a3-7145-45fe-b580-00a6ec4cc600 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:27.542253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9505f7-b2e2-4d76-bf2d-46fa9641bcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:27.542913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9505f7-b2e2-4d76-bf2d-46fa9641bcaf admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:27.543197 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9505f7-b2e2-4d76-bf2d-46fa9641bcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:27.543483 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9505f7-b2e2-4d76-bf2d-46fa9641bcaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:27.544260 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9505f7-b2e2-4d76-bf2d-46fa9641bcaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:27.546568 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9505f7-b2e2-4d76-bf2d-46fa9641bcaf admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:27.562386 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:27.594178 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1001/2015] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:34:27 2022] POST /identity/v3/projects => generated 372 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:34:27.594178 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9505f7-b2e2-4d76-bf2d-46fa9641bcaf admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:27.636510 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-dbd695a3-7145-45fe-b580-00a6ec4cc600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e204358f8e714395b14b51419eadc071'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:27.637601 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-dbd695a3-7145-45fe-b580-00a6ec4cc600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e204358f8e714395b14b51419eadc071'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:27.666302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-dbd695a3-7145-45fe-b580-00a6ec4cc600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e204358f8e714395b14b51419eadc071'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:27.680672 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-dbd695a3-7145-45fe-b580-00a6ec4cc600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e204358f8e714395b14b51419eadc071'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:27.696293 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-dbd695a3-7145-45fe-b580-00a6ec4cc600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e204358f8e714395b14b51419eadc071'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:27.707174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1015/2016] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:34:27 2022] DELETE /identity/v3/users/e204358f8e714395b14b51419eadc071 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:27.709983 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd695a3-7145-45fe-b580-00a6ec4cc600 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:27.715721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-dbd695a3-7145-45fe-b580-00a6ec4cc600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:27.725635 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9505f7-b2e2-4d76-bf2d-46fa9641bcaf admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:27.774060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:27.806534 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab7187-64a0-48bb-977d-a92ef3fada3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:27.807766 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab7187-64a0-48bb-977d-a92ef3fada3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:27.808304 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab7187-64a0-48bb-977d-a92ef3fada3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:27.808820 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab7187-64a0-48bb-977d-a92ef3fada3e admin admin] PATH_INFO: `/v3/projects/b283abe8db234842baaf93ce6e0acb7e/users/01bd5f36928c44e386350f854d5c2a81/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:27.825272 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd695a3-7145-45fe-b580-00a6ec4cc600 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:27.832304 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90baeca0-71ea-451c-a15b-5fbb6d8cc1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:27.833653 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90baeca0-71ea-451c-a15b-5fbb6d8cc1ea admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:27.833953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90baeca0-71ea-451c-a15b-5fbb6d8cc1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:27.834260 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90baeca0-71ea-451c-a15b-5fbb6d8cc1ea admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:27.835280 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90baeca0-71ea-451c-a15b-5fbb6d8cc1ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:27.837347 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90baeca0-71ea-451c-a15b-5fbb6d8cc1ea admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:27.838277 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab7187-64a0-48bb-977d-a92ef3fada3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b283abe8db234842baaf93ce6e0acb7e, user_id=01bd5f36928c44e386350f854d5c2a81, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:27.843936 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab7187-64a0-48bb-977d-a92ef3fada3e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:27.894939 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1002/2017] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:34:27 2022] PUT /identity/v3/projects/b283abe8db234842baaf93ce6e0acb7e/users/01bd5f36928c44e386350f854d5c2a81/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:27.899445 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab7187-64a0-48bb-977d-a92ef3fada3e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:27.904339 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1016/2018] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:34:27 2022] POST /identity/v3/users => generated 286 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:34:27.905968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-45ab7187-64a0-48bb-977d-a92ef3fada3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:27.906805 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9f04e-417c-4620-9ea3-cfb262cea5da None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:27.907330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9f04e-417c-4620-9ea3-cfb262cea5da None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:27.907748 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9f04e-417c-4620-9ea3-cfb262cea5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:27.942857 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-3da9f04e-417c-4620-9ea3-cfb262cea5da None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:27.944786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-3da9f04e-417c-4620-9ea3-cfb262cea5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:28.019355 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab7187-64a0-48bb-977d-a92ef3fada3e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:28.027764 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1017/2019] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:27 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:28.029171 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9f04e-417c-4620-9ea3-cfb262cea5da None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:28.031351 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-3da9f04e-417c-4620-9ea3-cfb262cea5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:28.040597 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cab921-f28e-4910-a024-6befe0792273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:28.041276 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cab921-f28e-4910-a024-6befe0792273 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:28.041565 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cab921-f28e-4910-a024-6befe0792273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:28.043134 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cab921-f28e-4910-a024-6befe0792273 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:28.043798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cab921-f28e-4910-a024-6befe0792273 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:28.046077 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cab921-f28e-4910-a024-6befe0792273 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:28.047610 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9f04e-417c-4620-9ea3-cfb262cea5da None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:28.054629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c879838-85e2-4816-a9b7-116199636204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:28.055660 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c879838-85e2-4816-a9b7-116199636204 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:28.056304 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c879838-85e2-4816-a9b7-116199636204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:28.056810 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c879838-85e2-4816-a9b7-116199636204 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:28.057497 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c879838-85e2-4816-a9b7-116199636204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:28.059439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c879838-85e2-4816-a9b7-116199636204 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:28.084145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1018/2020] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:34:28 2022] GET /identity/v3/roles => generated 1427 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:34:28.085341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c879838-85e2-4816-a9b7-116199636204 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:28.101492 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c879838-85e2-4816-a9b7-116199636204 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:28.110182 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1003/2021] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:34:27 2022] POST /identity/v3/users => generated 287 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:34:28.110515 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cdb384-319f-4d5b-973c-ba4582101e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:28.110736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cab921-f28e-4910-a024-6befe0792273 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:28.110900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdb384-319f-4d5b-973c-ba4582101e56 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:28.110900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdb384-319f-4d5b-973c-ba4582101e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:28.110900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdb384-319f-4d5b-973c-ba4582101e56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:28.110900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cdb384-319f-4d5b-973c-ba4582101e56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:28.111488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-22cab921-f28e-4910-a024-6befe0792273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:28.111664 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cdb384-319f-4d5b-973c-ba4582101e56 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:28.114724 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:28.131168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1019/2022] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:34:28 2022] GET /identity/v3/roles => generated 1427 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:34:28.134122 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cdb384-319f-4d5b-973c-ba4582101e56 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:28.153666 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cdb384-319f-4d5b-973c-ba4582101e56 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:28.159242 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d817fe-43a8-4421-a64c-6b3691a47470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:28.160206 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d817fe-43a8-4421-a64c-6b3691a47470 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:28.160661 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d817fe-43a8-4421-a64c-6b3691a47470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:28.161219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d817fe-43a8-4421-a64c-6b3691a47470 admin admin] PATH_INFO: `/v3/projects/b283abe8db234842baaf93ce6e0acb7e/users/01bd5f36928c44e386350f854d5c2a81/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:28.163766 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d817fe-43a8-4421-a64c-6b3691a47470 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b283abe8db234842baaf93ce6e0acb7e, user_id=01bd5f36928c44e386350f854d5c2a81, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:28.167346 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d817fe-43a8-4421-a64c-6b3691a47470 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:28.182615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1020/2023] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:34:28 2022] PUT /identity/v3/projects/b283abe8db234842baaf93ce6e0acb7e/users/01bd5f36928c44e386350f854d5c2a81/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:28.183714 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d817fe-43a8-4421-a64c-6b3691a47470 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:28.243107 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cab921-f28e-4910-a024-6befe0792273 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:28.304050 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406936b-552a-409c-a4eb-4bf33dccbe64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:28.304974 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406936b-552a-409c-a4eb-4bf33dccbe64 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:28.305443 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406936b-552a-409c-a4eb-4bf33dccbe64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:28.305806 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406936b-552a-409c-a4eb-4bf33dccbe64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:28.322056 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d817fe-43a8-4421-a64c-6b3691a47470 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:28.330747 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ae3b3-e249-48e0-9c08-c1f2fe5ef678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:28.332121 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ae3b3-e249-48e0-9c08-c1f2fe5ef678 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:28.332611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ae3b3-e249-48e0-9c08-c1f2fe5ef678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:28.333106 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ae3b3-e249-48e0-9c08-c1f2fe5ef678 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:28.334308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6ae3b3-e249-48e0-9c08-c1f2fe5ef678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:28.336354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6ae3b3-e249-48e0-9c08-c1f2fe5ef678 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:28.351094 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1021/2024] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:34:28 2022] GET /identity/v3/roles => generated 1427 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:34:28.352671 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:28.354552 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ae3b3-e249-48e0-9c08-c1f2fe5ef678 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:28.372154 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ae3b3-e249-48e0-9c08-c1f2fe5ef678 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:28.382869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca032759-0da1-4772-abf5-76414723e77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:28.383721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca032759-0da1-4772-abf5-76414723e77e admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:28.384043 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca032759-0da1-4772-abf5-76414723e77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:28.384329 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca032759-0da1-4772-abf5-76414723e77e admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:28.385365 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca032759-0da1-4772-abf5-76414723e77e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:28.387082 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca032759-0da1-4772-abf5-76414723e77e admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:28.399546 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1022/2025] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:34:28 2022] GET /identity/v3/roles => generated 1427 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:34:28.401366 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918afcf-7b84-4312-af4e-40da0fe0feb6 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:28.401665 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918afcf-7b84-4312-af4e-40da0fe0feb6 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:28.401952 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918afcf-7b84-4312-af4e-40da0fe0feb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:28.428451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406936b-552a-409c-a4eb-4bf33dccbe64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:28.429967 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406936b-552a-409c-a4eb-4bf33dccbe64 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:28.439816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-9918afcf-7b84-4312-af4e-40da0fe0feb6 None None] MFA Rules not processed for user `01bd5f36928c44e386350f854d5c2a81`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:28.441984 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1004/2026] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:34:28 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:28.442220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-9918afcf-7b84-4312-af4e-40da0fe0feb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:28.443527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406936b-552a-409c-a4eb-4bf33dccbe64 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:28.453881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406936b-552a-409c-a4eb-4bf33dccbe64 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:28.459831 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b710dc-fb2e-4b2a-87f3-af2b355f1138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:28.460552 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b710dc-fb2e-4b2a-87f3-af2b355f1138 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:28.460835 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b710dc-fb2e-4b2a-87f3-af2b355f1138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:28.461118 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b710dc-fb2e-4b2a-87f3-af2b355f1138 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:28.462073 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b710dc-fb2e-4b2a-87f3-af2b355f1138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:28.464470 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b710dc-fb2e-4b2a-87f3-af2b355f1138 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:28.478202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1005/2027] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:34:28 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:34:28.479213 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b710dc-fb2e-4b2a-87f3-af2b355f1138 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:28.498938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b710dc-fb2e-4b2a-87f3-af2b355f1138 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:28.510815 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fddec0d-c029-4e25-b4dc-24f72ce5d7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:28.514323 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fddec0d-c029-4e25-b4dc-24f72ce5d7ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:28.515603 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fddec0d-c029-4e25-b4dc-24f72ce5d7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:28.516551 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fddec0d-c029-4e25-b4dc-24f72ce5d7ba admin admin] PATH_INFO: `/v3/projects/891d22474d19456ab70b881060dadde0/users/a6734615dd7f4c12bbe2e0acec7e24da/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:28.552587 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fddec0d-c029-4e25-b4dc-24f72ce5d7ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=891d22474d19456ab70b881060dadde0, user_id=a6734615dd7f4c12bbe2e0acec7e24da, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:28.558956 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fddec0d-c029-4e25-b4dc-24f72ce5d7ba admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:28.579121 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1023/2028] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:28 2022] POST /identity/v3/auth/tokens => generated 3361 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:28.579121 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9918afcf-7b84-4312-af4e-40da0fe0feb6 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:28.603280 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9918afcf-7b84-4312-af4e-40da0fe0feb6 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:28.614723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1006/2029] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:34:28 2022] PUT /identity/v3/projects/891d22474d19456ab70b881060dadde0/users/a6734615dd7f4c12bbe2e0acec7e24da/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:28.616518 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fddec0d-c029-4e25-b4dc-24f72ce5d7ba admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:28.619911 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5fddec0d-c029-4e25-b4dc-24f72ce5d7ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:28.789925 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:28.820060 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ec12e-1c3c-44f9-a127-be9fc051ac13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:28.821124 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ec12e-1c3c-44f9-a127-be9fc051ac13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:28.821408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ec12e-1c3c-44f9-a127-be9fc051ac13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:28.821690 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ec12e-1c3c-44f9-a127-be9fc051ac13 admin admin] PATH_INFO: `/v3/projects/7cf9298f363f46aa860b2bceef00d8a7/users/078c26677e0340899511ac3af69544ff/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:28.826530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fddec0d-c029-4e25-b4dc-24f72ce5d7ba admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:28.834095 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a45626-3dd2-4095-8795-4fa502521089 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:28.834759 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a45626-3dd2-4095-8795-4fa502521089 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:28.835039 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a45626-3dd2-4095-8795-4fa502521089 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:28.835313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a45626-3dd2-4095-8795-4fa502521089 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:28.863693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ec12e-1c3c-44f9-a127-be9fc051ac13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cf9298f363f46aa860b2bceef00d8a7, user_id=078c26677e0340899511ac3af69544ff, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:28.880599 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ec12e-1c3c-44f9-a127-be9fc051ac13 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:28.923134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1024/2030] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:34:28 2022] PUT /identity/v3/projects/7cf9298f363f46aa860b2bceef00d8a7/users/078c26677e0340899511ac3af69544ff/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:28.924702 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46241d1-8f1c-4767-8982-6f3e423bf147 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:28.924991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46241d1-8f1c-4767-8982-6f3e423bf147 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:28.925271 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46241d1-8f1c-4767-8982-6f3e423bf147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:29.035777 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-b46241d1-8f1c-4767-8982-6f3e423bf147 None None] MFA Rules not processed for user `a6734615dd7f4c12bbe2e0acec7e24da`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:29.037284 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-b46241d1-8f1c-4767-8982-6f3e423bf147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:29.064529 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a45626-3dd2-4095-8795-4fa502521089 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:29.067925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a45626-3dd2-4095-8795-4fa502521089 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:29.088737 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1007/2031] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:34:28 2022] GET /identity/v3/auth/tokens => generated 3361 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:29.091197 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeecda5-fe8c-4ca6-8a93-53dac4e3710f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:29.091671 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeecda5-fe8c-4ca6-8a93-53dac4e3710f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:29.092264 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeecda5-fe8c-4ca6-8a93-53dac4e3710f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:29.136533 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1025/2032] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:29.137757 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46241d1-8f1c-4767-8982-6f3e423bf147 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:29.144436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-b46241d1-8f1c-4767-8982-6f3e423bf147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:29.161177 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-bbeecda5-fe8c-4ca6-8a93-53dac4e3710f None None] MFA Rules not processed for user `078c26677e0340899511ac3af69544ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:29.164123 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bbeecda5-fe8c-4ca6-8a93-53dac4e3710f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:29.208225 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:29.258652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46241d1-8f1c-4767-8982-6f3e423bf147 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:29.263839 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1008/2033] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:29.266057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeecda5-fe8c-4ca6-8a93-53dac4e3710f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:29.270651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee5241-3171-492b-ba2e-60f83006dd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:29.271765 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee5241-3171-492b-ba2e-60f83006dd33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:29.272505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee5241-3171-492b-ba2e-60f83006dd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:29.274094 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee5241-3171-492b-ba2e-60f83006dd33 admin admin] PATH_INFO: `/v3/projects/2439b61aa6b7466288b1e4b03a87c5e8` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:29.277528 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee5241-3171-492b-ba2e-60f83006dd33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2439b61aa6b7466288b1e4b03a87c5e8)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:29.280392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee5241-3171-492b-ba2e-60f83006dd33 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:29.282116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bbeecda5-fe8c-4ca6-8a93-53dac4e3710f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:29.382387 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeecda5-fe8c-4ca6-8a93-53dac4e3710f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:29.394528 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a6ee5241-3171-492b-ba2e-60f83006dd33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2439b61aa6b7466288b1e4b03a87c5e8', 'request_id': 'req-a6ee5241-3171-492b-ba2e-60f83006dd33'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:29.413358 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a6ee5241-3171-492b-ba2e-60f83006dd33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2439b61aa6b7466288b1e4b03a87c5e8', 'request_id': 'req-a6ee5241-3171-492b-ba2e-60f83006dd33'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:29.430614 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a6ee5241-3171-492b-ba2e-60f83006dd33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2439b61aa6b7466288b1e4b03a87c5e8', 'request_id': 'req-a6ee5241-3171-492b-ba2e-60f83006dd33'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:29.447033 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:29.459102 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a6ee5241-3171-492b-ba2e-60f83006dd33 admin admin] The token cache is being invalidate because project 2439b61aa6b7466288b1e4b03a87c5e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:34:29.460448 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a6ee5241-3171-492b-ba2e-60f83006dd33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:29.462720 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a6ee5241-3171-492b-ba2e-60f83006dd33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:29.465749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1026/2034] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:34:29 2022] DELETE /identity/v3/projects/2439b61aa6b7466288b1e4b03a87c5e8 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:29.467790 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee5241-3171-492b-ba2e-60f83006dd33 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:29.470686 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-a6ee5241-3171-492b-ba2e-60f83006dd33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:29.507657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f401c66-3b59-424d-aa26-a99034256bd1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b31c86bc9334d2b8acc35a035f90442', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:29.512661 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f401c66-3b59-424d-aa26-a99034256bd1 service glance] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:29.513380 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f401c66-3b59-424d-aa26-a99034256bd1 service glance] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:29.514597 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f401c66-3b59-424d-aa26-a99034256bd1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:29.518590 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6f401c66-3b59-424d-aa26-a99034256bd1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:29.594975 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee5241-3171-492b-ba2e-60f83006dd33 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:29.599909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337749eb-ceac-406f-9f30-9865414de983 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b31c86bc9334d2b8acc35a035f90442', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:29.602047 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337749eb-ceac-406f-9f30-9865414de983 service glance] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:29.602910 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337749eb-ceac-406f-9f30-9865414de983 service glance] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:29.603274 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337749eb-ceac-406f-9f30-9865414de983 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:29.606005 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-337749eb-ceac-406f-9f30-9865414de983 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:29.638338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f401c66-3b59-424d-aa26-a99034256bd1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:29.641118 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f401c66-3b59-424d-aa26-a99034256bd1 service glance] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:29.655690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1009/2035] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:34:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:29.657824 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f401c66-3b59-424d-aa26-a99034256bd1 service glance] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:29.659562 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6f401c66-3b59-424d-aa26-a99034256bd1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:29.737131 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337749eb-ceac-406f-9f30-9865414de983 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:29.740463 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337749eb-ceac-406f-9f30-9865414de983 service glance] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:29.744276 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:29.762568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1027/2036] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:34:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:29.765241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af076d-24f7-481e-aa88-8a6a9997973c None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:29.765663 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af076d-24f7-481e-aa88-8a6a9997973c None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:29.766132 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af076d-24f7-481e-aa88-8a6a9997973c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:29.807201 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-09af076d-24f7-481e-aa88-8a6a9997973c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:29.810656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-09af076d-24f7-481e-aa88-8a6a9997973c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:29.811897 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f401c66-3b59-424d-aa26-a99034256bd1 service glance] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:29.824448 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f8dab-66f3-4d32-a15f-7c14b972265d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:29.826229 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f8dab-66f3-4d32-a15f-7c14b972265d service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:29.826229 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f8dab-66f3-4d32-a15f-7c14b972265d service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:29.826229 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f8dab-66f3-4d32-a15f-7c14b972265d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:29.830286 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ce9f8dab-66f3-4d32-a15f-7c14b972265d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:29.995903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f8dab-66f3-4d32-a15f-7c14b972265d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:29.999104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f8dab-66f3-4d32-a15f-7c14b972265d service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:30.019827 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1028/2037] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:29 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:30.022533 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1010/2038] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:34:29 2022] GET /identity/v3/auth/tokens => generated 3294 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:30.045535 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f93ae-1a1a-419b-b05d-e7a24bbf3241 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:30.046386 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f93ae-1a1a-419b-b05d-e7a24bbf3241 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:30.047038 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f93ae-1a1a-419b-b05d-e7a24bbf3241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:30.120306 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-672f93ae-1a1a-419b-b05d-e7a24bbf3241 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:30.122766 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-672f93ae-1a1a-419b-b05d-e7a24bbf3241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:30.154009 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1029/2039] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:30.155374 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f8dab-66f3-4d32-a15f-7c14b972265d service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:30.177209 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f8dab-66f3-4d32-a15f-7c14b972265d service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:30.197243 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35094647-bda4-4384-9d46-7f18fa286ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:30.200053 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35094647-bda4-4384-9d46-7f18fa286ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:30.201059 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35094647-bda4-4384-9d46-7f18fa286ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:30.201640 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35094647-bda4-4384-9d46-7f18fa286ae5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:30.203220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35094647-bda4-4384-9d46-7f18fa286ae5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:34:30.204819 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35094647-bda4-4384-9d46-7f18fa286ae5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:30.209311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35094647-bda4-4384-9d46-7f18fa286ae5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:30.266043 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1011/2040] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:34:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:34:30.267432 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f93ae-1a1a-419b-b05d-e7a24bbf3241 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:30.294899 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f93ae-1a1a-419b-b05d-e7a24bbf3241 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:30.305052 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939a54d-d259-4e23-90bc-a0d15df2be63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:30.306152 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939a54d-d259-4e23-90bc-a0d15df2be63 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:30.309319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939a54d-d259-4e23-90bc-a0d15df2be63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:30.309961 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939a54d-d259-4e23-90bc-a0d15df2be63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:30.311684 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939a54d-d259-4e23-90bc-a0d15df2be63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:30.317619 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939a54d-d259-4e23-90bc-a0d15df2be63 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:30.396094 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1030/2041] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:34:30 2022] POST /identity/v3/projects => generated 386 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:34:30.404036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35094647-bda4-4384-9d46-7f18fa286ae5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:30.430193 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35094647-bda4-4384-9d46-7f18fa286ae5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:30.555871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532cb0c-d7ae-484a-8d83-2b2d50095893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:30.557138 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532cb0c-d7ae-484a-8d83-2b2d50095893 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:30.568000 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532cb0c-d7ae-484a-8d83-2b2d50095893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:30.568000 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532cb0c-d7ae-484a-8d83-2b2d50095893 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:30.568000 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4532cb0c-d7ae-484a-8d83-2b2d50095893 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:30.568000 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4532cb0c-d7ae-484a-8d83-2b2d50095893 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:30.604464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1012/2042] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:34:30 2022] POST /identity/v3/users => generated 294 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:34:30.609937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939a54d-d259-4e23-90bc-a0d15df2be63 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:30.637315 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939a54d-d259-4e23-90bc-a0d15df2be63 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:30.663837 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810dab8-dc20-41dd-a98a-da064e05b64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:30.664777 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810dab8-dc20-41dd-a98a-da064e05b64e admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:30.665205 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810dab8-dc20-41dd-a98a-da064e05b64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:30.665764 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810dab8-dc20-41dd-a98a-da064e05b64e admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:30.673531 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7810dab8-dc20-41dd-a98a-da064e05b64e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:30.678591 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7810dab8-dc20-41dd-a98a-da064e05b64e admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:30.704337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1031/2043] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:34:30 2022] GET /identity/v3/roles => generated 1427 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:34:30.710967 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532cb0c-d7ae-484a-8d83-2b2d50095893 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:30.738981 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532cb0c-d7ae-484a-8d83-2b2d50095893 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:30.757838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150053c-d839-42fa-8ba4-708054a0ed39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:30.762410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150053c-d839-42fa-8ba4-708054a0ed39 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:30.763043 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150053c-d839-42fa-8ba4-708054a0ed39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:30.764334 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150053c-d839-42fa-8ba4-708054a0ed39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:30.765494 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1150053c-d839-42fa-8ba4-708054a0ed39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:30.780189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1150053c-d839-42fa-8ba4-708054a0ed39 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:30.811077 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1013/2044] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:34:30 2022] GET /identity/v3/roles => generated 1427 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:34:30.814595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810dab8-dc20-41dd-a98a-da064e05b64e admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:30.860200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810dab8-dc20-41dd-a98a-da064e05b64e admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:30.872045 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148512e8-695f-4a94-8a20-e98a339a23e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:30.873107 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148512e8-695f-4a94-8a20-e98a339a23e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:30.873715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148512e8-695f-4a94-8a20-e98a339a23e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:30.873988 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148512e8-695f-4a94-8a20-e98a339a23e0 admin admin] PATH_INFO: `/v3/projects/24a124fa08754cf48f3142de377ab508/users/b03e5adc812f45dca09ff634dc297a76/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:30.901840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:30.910828 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148512e8-695f-4a94-8a20-e98a339a23e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24a124fa08754cf48f3142de377ab508, user_id=b03e5adc812f45dca09ff634dc297a76, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:30.918784 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148512e8-695f-4a94-8a20-e98a339a23e0 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:30.951209 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1032/2045] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:34:30 2022] PUT /identity/v3/projects/24a124fa08754cf48f3142de377ab508/users/b03e5adc812f45dca09ff634dc297a76/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:30.964810 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10957a78-98d0-4a59-b1ed-01e09af116ee None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:30.965276 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10957a78-98d0-4a59-b1ed-01e09af116ee None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:30.966163 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10957a78-98d0-4a59-b1ed-01e09af116ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:31.005523 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-10957a78-98d0-4a59-b1ed-01e09af116ee None None] MFA Rules not processed for user `b03e5adc812f45dca09ff634dc297a76`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:31.006876 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-10957a78-98d0-4a59-b1ed-01e09af116ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:31.083291 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:31.120197 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1014/2046] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:30 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:31.133840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148512e8-695f-4a94-8a20-e98a339a23e0 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:31.137596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-148512e8-695f-4a94-8a20-e98a339a23e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:31.225055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148512e8-695f-4a94-8a20-e98a339a23e0 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:31.233316 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb9c4d-7de9-478e-b83c-6e5fd51ce0d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:31.234977 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb9c4d-7de9-478e-b83c-6e5fd51ce0d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:31.235712 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb9c4d-7de9-478e-b83c-6e5fd51ce0d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:31.236248 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb9c4d-7de9-478e-b83c-6e5fd51ce0d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:31.254960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb9c4d-7de9-478e-b83c-6e5fd51ce0d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:31.259351 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb9c4d-7de9-478e-b83c-6e5fd51ce0d8 service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:31.274820 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1033/2047] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:34:31 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:34:41.113843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=7 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:41.267922 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:51.977419 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10957a78-98d0-4a59-b1ed-01e09af116ee None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:51.979252 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-10957a78-98d0-4a59-b1ed-01e09af116ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:52.059756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10957a78-98d0-4a59-b1ed-01e09af116ee None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:52.066256 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61780e-ecb8-482d-9ae4-cc8b611ef91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:52.066936 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61780e-ecb8-482d-9ae4-cc8b611ef91b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:52.067305 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61780e-ecb8-482d-9ae4-cc8b611ef91b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:52.067502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61780e-ecb8-482d-9ae4-cc8b611ef91b admin admin] PATH_INFO: `/v3/users/ee37026a989a40bebb2dd6bfeee4193d` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:52.068885 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61780e-ecb8-482d-9ae4-cc8b611ef91b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee37026a989a40bebb2dd6bfeee4193d)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:52.070870 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61780e-ecb8-482d-9ae4-cc8b611ef91b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:52.123732 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fc61780e-ecb8-482d-9ae4-cc8b611ef91b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee37026a989a40bebb2dd6bfeee4193d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:52.131700 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fc61780e-ecb8-482d-9ae4-cc8b611ef91b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee37026a989a40bebb2dd6bfeee4193d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:52.132404 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fc61780e-ecb8-482d-9ae4-cc8b611ef91b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee37026a989a40bebb2dd6bfeee4193d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:52.145755 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fc61780e-ecb8-482d-9ae4-cc8b611ef91b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee37026a989a40bebb2dd6bfeee4193d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:52.146559 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fc61780e-ecb8-482d-9ae4-cc8b611ef91b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee37026a989a40bebb2dd6bfeee4193d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:52.168373 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1015/2048] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:34:51 2022] DELETE /identity/v3/users/ee37026a989a40bebb2dd6bfeee4193d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:52.173896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4171c7e-f4cd-465d-a6ed-3da403f396cc None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:52.174536 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4171c7e-f4cd-465d-a6ed-3da403f396cc None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:52.174536 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4171c7e-f4cd-465d-a6ed-3da403f396cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:52.198277 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-f4171c7e-f4cd-465d-a6ed-3da403f396cc None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:52.199536 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-f4171c7e-f4cd-465d-a6ed-3da403f396cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:52.262937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1034/2049] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:52.275241 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61780e-ecb8-482d-9ae4-cc8b611ef91b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:52.277013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fc61780e-ecb8-482d-9ae4-cc8b611ef91b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:52.346329 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61780e-ecb8-482d-9ae4-cc8b611ef91b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:52.350788 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0090a2-c44a-4a0e-b4af-5131fbe4d062 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:52.351491 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0090a2-c44a-4a0e-b4af-5131fbe4d062 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:52.351779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0090a2-c44a-4a0e-b4af-5131fbe4d062 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:52.352454 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0090a2-c44a-4a0e-b4af-5131fbe4d062 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:52.362661 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0090a2-c44a-4a0e-b4af-5131fbe4d062 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:52.364195 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0090a2-c44a-4a0e-b4af-5131fbe4d062 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:52.374071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1016/2050] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:34:52 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:52.725790 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4171c7e-f4cd-465d-a6ed-3da403f396cc None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:52.728589 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-f4171c7e-f4cd-465d-a6ed-3da403f396cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:52.745926 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4171c7e-f4cd-465d-a6ed-3da403f396cc None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:52.754310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb0278-57d8-475d-9381-9eab5c304c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:52.755217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb0278-57d8-475d-9381-9eab5c304c2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:52.756072 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb0278-57d8-475d-9381-9eab5c304c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:52.756761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb0278-57d8-475d-9381-9eab5c304c2a admin admin] PATH_INFO: `/v3/projects/dd26e858de04440fada96f1845a16cf4` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:52.759032 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb0278-57d8-475d-9381-9eab5c304c2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd26e858de04440fada96f1845a16cf4)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:52.762027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb0278-57d8-475d-9381-9eab5c304c2a admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:52.824864 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6cbb0278-57d8-475d-9381-9eab5c304c2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd26e858de04440fada96f1845a16cf4', 'request_id': 'req-6cbb0278-57d8-475d-9381-9eab5c304c2a'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:52.835465 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6cbb0278-57d8-475d-9381-9eab5c304c2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd26e858de04440fada96f1845a16cf4', 'request_id': 'req-6cbb0278-57d8-475d-9381-9eab5c304c2a'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:52.845487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6cbb0278-57d8-475d-9381-9eab5c304c2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd26e858de04440fada96f1845a16cf4', 'request_id': 'req-6cbb0278-57d8-475d-9381-9eab5c304c2a'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:52.863039 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6cbb0278-57d8-475d-9381-9eab5c304c2a admin admin] The token cache is being invalidate because project dd26e858de04440fada96f1845a16cf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:34:52.863508 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6cbb0278-57d8-475d-9381-9eab5c304c2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:52.864472 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6cbb0278-57d8-475d-9381-9eab5c304c2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:52.866361 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1035/2051] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:34:52 2022] DELETE /identity/v3/projects/dd26e858de04440fada96f1845a16cf4 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:52.867048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0090a2-c44a-4a0e-b4af-5131fbe4d062 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:52.869166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ec0090a2-c44a-4a0e-b4af-5131fbe4d062 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:52.870358 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb0278-57d8-475d-9381-9eab5c304c2a admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:52.873047 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6cbb0278-57d8-475d-9381-9eab5c304c2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:52.884043 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0090a2-c44a-4a0e-b4af-5131fbe4d062 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:52.913141 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:52.961890 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb0278-57d8-475d-9381-9eab5c304c2a admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:52.966051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c2700-18a5-46e1-8286-278b662a4db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:52.966756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c2700-18a5-46e1-8286-278b662a4db3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:52.967016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c2700-18a5-46e1-8286-278b662a4db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:52.967314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c2700-18a5-46e1-8286-278b662a4db3 admin admin] PATH_INFO: `/v3/users/b03e5adc812f45dca09ff634dc297a76` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:52.967611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aecd25d-8af0-4e94-9f7e-4280e8d61c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:52.968679 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecd25d-8af0-4e94-9f7e-4280e8d61c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:52.968872 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c2700-18a5-46e1-8286-278b662a4db3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b03e5adc812f45dca09ff634dc297a76)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:52.969135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecd25d-8af0-4e94-9f7e-4280e8d61c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:52.969531 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecd25d-8af0-4e94-9f7e-4280e8d61c37 admin admin] PATH_INFO: `/v3/users/01bd5f36928c44e386350f854d5c2a81` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:52.970706 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c2700-18a5-46e1-8286-278b662a4db3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:52.970988 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aecd25d-8af0-4e94-9f7e-4280e8d61c37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01bd5f36928c44e386350f854d5c2a81)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:52.973559 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aecd25d-8af0-4e94-9f7e-4280e8d61c37 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:53.005173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:53.029718 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9c9c2700-18a5-46e1-8286-278b662a4db3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b03e5adc812f45dca09ff634dc297a76'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:53.032457 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-2aecd25d-8af0-4e94-9f7e-4280e8d61c37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01bd5f36928c44e386350f854d5c2a81'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:53.033411 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-2aecd25d-8af0-4e94-9f7e-4280e8d61c37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01bd5f36928c44e386350f854d5c2a81'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:53.038370 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9c9c2700-18a5-46e1-8286-278b662a4db3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b03e5adc812f45dca09ff634dc297a76'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:53.039117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9c9c2700-18a5-46e1-8286-278b662a4db3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b03e5adc812f45dca09ff634dc297a76'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:53.046722 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-2aecd25d-8af0-4e94-9f7e-4280e8d61c37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01bd5f36928c44e386350f854d5c2a81'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:53.053813 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9c9c2700-18a5-46e1-8286-278b662a4db3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b03e5adc812f45dca09ff634dc297a76'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:53.054027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-2aecd25d-8af0-4e94-9f7e-4280e8d61c37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01bd5f36928c44e386350f854d5c2a81'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:53.054576 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9c9c2700-18a5-46e1-8286-278b662a4db3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b03e5adc812f45dca09ff634dc297a76'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:53.063733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-2aecd25d-8af0-4e94-9f7e-4280e8d61c37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01bd5f36928c44e386350f854d5c2a81'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:53.075027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1036/2052] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:34:52 2022] DELETE /identity/v3/users/01bd5f36928c44e386350f854d5c2a81 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:53.083693 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1017/2053] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:34:52 2022] DELETE /identity/v3/users/b03e5adc812f45dca09ff634dc297a76 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:53.090371 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3ff60-77a0-4da6-890b-154cf3ebabbf None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:53.090829 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3ff60-77a0-4da6-890b-154cf3ebabbf None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:53.091295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3ff60-77a0-4da6-890b-154cf3ebabbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:53.123922 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-d2d3ff60-77a0-4da6-890b-154cf3ebabbf None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:53.125130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d2d3ff60-77a0-4da6-890b-154cf3ebabbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:53.201689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1037/2054] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:53.211467 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c2700-18a5-46e1-8286-278b662a4db3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:53.213071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9c9c2700-18a5-46e1-8286-278b662a4db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:53.300274 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c2700-18a5-46e1-8286-278b662a4db3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:53.305234 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be330c-b8e4-48a3-a748-762bfd6ee8e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:53.305899 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be330c-b8e4-48a3-a748-762bfd6ee8e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:53.306190 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be330c-b8e4-48a3-a748-762bfd6ee8e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:53.306473 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be330c-b8e4-48a3-a748-762bfd6ee8e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:53.322875 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be330c-b8e4-48a3-a748-762bfd6ee8e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:53.324434 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be330c-b8e4-48a3-a748-762bfd6ee8e8 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:53.336714 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1018/2055] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:34:53 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:53.532789 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3ff60-77a0-4da6-890b-154cf3ebabbf None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:53.534291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d2d3ff60-77a0-4da6-890b-154cf3ebabbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:53.547254 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3ff60-77a0-4da6-890b-154cf3ebabbf None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:53.553549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf4505-5d33-494a-bdd4-296795d6c401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:53.553765 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf4505-5d33-494a-bdd4-296795d6c401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:53.554047 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf4505-5d33-494a-bdd4-296795d6c401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:53.554330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf4505-5d33-494a-bdd4-296795d6c401 admin admin] PATH_INFO: `/v3/projects/b283abe8db234842baaf93ce6e0acb7e` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:53.555612 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf4505-5d33-494a-bdd4-296795d6c401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b283abe8db234842baaf93ce6e0acb7e)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:53.557685 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf4505-5d33-494a-bdd4-296795d6c401 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:53.625328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-efaf4505-5d33-494a-bdd4-296795d6c401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b283abe8db234842baaf93ce6e0acb7e', 'request_id': 'req-efaf4505-5d33-494a-bdd4-296795d6c401'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:53.635203 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-efaf4505-5d33-494a-bdd4-296795d6c401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b283abe8db234842baaf93ce6e0acb7e', 'request_id': 'req-efaf4505-5d33-494a-bdd4-296795d6c401'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:53.645121 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-efaf4505-5d33-494a-bdd4-296795d6c401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b283abe8db234842baaf93ce6e0acb7e', 'request_id': 'req-efaf4505-5d33-494a-bdd4-296795d6c401'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:53.658533 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-efaf4505-5d33-494a-bdd4-296795d6c401 admin admin] The token cache is being invalidate because project b283abe8db234842baaf93ce6e0acb7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:34:53.658787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-efaf4505-5d33-494a-bdd4-296795d6c401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:53.659601 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-efaf4505-5d33-494a-bdd4-296795d6c401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:53.661337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1038/2056] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:34:53 2022] DELETE /identity/v3/projects/b283abe8db234842baaf93ce6e0acb7e => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:53.668658 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281c50c-eaff-486c-8c18-7c65265b9958 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:53.668944 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281c50c-eaff-486c-8c18-7c65265b9958 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:53.669294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281c50c-eaff-486c-8c18-7c65265b9958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:53.693157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0281c50c-eaff-486c-8c18-7c65265b9958 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:53.694702 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0281c50c-eaff-486c-8c18-7c65265b9958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:53.713271 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf4505-5d33-494a-bdd4-296795d6c401 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:53.715061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-efaf4505-5d33-494a-bdd4-296795d6c401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:53.752170 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:53.776488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1019/2057] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:34:53.790553 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05038d-900a-4e21-a893-af4203001fda None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:53.790940 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05038d-900a-4e21-a893-af4203001fda None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:53.791211 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05038d-900a-4e21-a893-af4203001fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:53.793761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:53.817727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9b05038d-900a-4e21-a893-af4203001fda None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:53.819348 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9b05038d-900a-4e21-a893-af4203001fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:53.827731 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1020/2058] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:53.832171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05038d-900a-4e21-a893-af4203001fda None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:53.837248 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf4505-5d33-494a-bdd4-296795d6c401 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:53.842301 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05038d-900a-4e21-a893-af4203001fda None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:53.843393 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be765b10-ecdd-411c-862c-942d3dc8c9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:53.844224 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be765b10-ecdd-411c-862c-942d3dc8c9a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:53.844575 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be765b10-ecdd-411c-862c-942d3dc8c9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:53.844912 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be765b10-ecdd-411c-862c-942d3dc8c9a8 admin admin] PATH_INFO: `/v3/projects/24a124fa08754cf48f3142de377ab508` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:53.847753 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a47d8-f4d9-422c-a5e3-d580dd274838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:53.848440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a47d8-f4d9-422c-a5e3-d580dd274838 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:53.848722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a47d8-f4d9-422c-a5e3-d580dd274838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:53.849007 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a47d8-f4d9-422c-a5e3-d580dd274838 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:53.849654 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be765b10-ecdd-411c-862c-942d3dc8c9a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24a124fa08754cf48f3142de377ab508)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:53.849879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a47d8-f4d9-422c-a5e3-d580dd274838 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:34:53.850121 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a47d8-f4d9-422c-a5e3-d580dd274838 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:53.852323 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be765b10-ecdd-411c-862c-942d3dc8c9a8 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:53.852629 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a47d8-f4d9-422c-a5e3-d580dd274838 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:53.869525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1021/2059] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:34:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:34:53.873200 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a47d8-f4d9-422c-a5e3-d580dd274838 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:53.884223 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a47d8-f4d9-422c-a5e3-d580dd274838 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:53.890594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c2abe-14a8-4d09-9930-3d9743ac6b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:53.891907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c2abe-14a8-4d09-9930-3d9743ac6b6a admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:53.891907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c2abe-14a8-4d09-9930-3d9743ac6b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:53.891907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c2abe-14a8-4d09-9930-3d9743ac6b6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:53.892455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c2abe-14a8-4d09-9930-3d9743ac6b6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:53.894381 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c2abe-14a8-4d09-9930-3d9743ac6b6a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:53.931922 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-be765b10-ecdd-411c-862c-942d3dc8c9a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24a124fa08754cf48f3142de377ab508', 'request_id': 'req-be765b10-ecdd-411c-862c-942d3dc8c9a8'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:53.934915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1022/2060] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:34:53 2022] POST /identity/v3/projects => generated 366 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:34:53.939661 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c2abe-14a8-4d09-9930-3d9743ac6b6a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:53.944330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-be765b10-ecdd-411c-862c-942d3dc8c9a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24a124fa08754cf48f3142de377ab508', 'request_id': 'req-be765b10-ecdd-411c-862c-942d3dc8c9a8'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:53.954210 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-be765b10-ecdd-411c-862c-942d3dc8c9a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24a124fa08754cf48f3142de377ab508', 'request_id': 'req-be765b10-ecdd-411c-862c-942d3dc8c9a8'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:53.973348 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-be765b10-ecdd-411c-862c-942d3dc8c9a8 admin admin] The token cache is being invalidate because project 24a124fa08754cf48f3142de377ab508 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:34:53.973657 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-be765b10-ecdd-411c-862c-942d3dc8c9a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:53.974399 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-be765b10-ecdd-411c-862c-942d3dc8c9a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:34:53.976320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1039/2061] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:34:53 2022] DELETE /identity/v3/projects/24a124fa08754cf48f3142de377ab508 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:53.981994 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcf620-32b2-4bb3-99dd-e1793e6c73b0 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:53.982263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcf620-32b2-4bb3-99dd-e1793e6c73b0 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:53.982546 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcf620-32b2-4bb3-99dd-e1793e6c73b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:54.006119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-28bcf620-32b2-4bb3-99dd-e1793e6c73b0 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:54.007405 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-28bcf620-32b2-4bb3-99dd-e1793e6c73b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:54.041374 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c2abe-14a8-4d09-9930-3d9743ac6b6a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:54.046733 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614aa4e5-c8da-4094-82f6-34466ce496b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:54.047391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614aa4e5-c8da-4094-82f6-34466ce496b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:54.047681 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614aa4e5-c8da-4094-82f6-34466ce496b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:54.048006 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614aa4e5-c8da-4094-82f6-34466ce496b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:54.048699 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614aa4e5-c8da-4094-82f6-34466ce496b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:54.050737 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614aa4e5-c8da-4094-82f6-34466ce496b6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:54.085533 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1023/2062] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:34:53 2022] POST /identity/v3/users => generated 284 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:34:54.088830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614aa4e5-c8da-4094-82f6-34466ce496b6 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:54.090890 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-614aa4e5-c8da-4094-82f6-34466ce496b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:54.101974 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1040/2063] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:54.108245 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614aa4e5-c8da-4094-82f6-34466ce496b6 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:54.116320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e62fe4f-dfd3-4f50-878f-ca54fdd69b44 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:54.116580 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19015bd1-6e29-460c-92cd-b968030f477c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:54.117013 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e62fe4f-dfd3-4f50-878f-ca54fdd69b44 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:54.117406 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19015bd1-6e29-460c-92cd-b968030f477c admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:54.117813 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e62fe4f-dfd3-4f50-878f-ca54fdd69b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:54.117920 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19015bd1-6e29-460c-92cd-b968030f477c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:54.118312 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19015bd1-6e29-460c-92cd-b968030f477c admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:54.119464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19015bd1-6e29-460c-92cd-b968030f477c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:54.122335 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19015bd1-6e29-460c-92cd-b968030f477c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:54.141563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1024/2064] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:34:54 2022] GET /identity/v3/roles => generated 1427 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:34:54.145622 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19015bd1-6e29-460c-92cd-b968030f477c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:54.154295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-2e62fe4f-dfd3-4f50-878f-ca54fdd69b44 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:54.155586 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-2e62fe4f-dfd3-4f50-878f-ca54fdd69b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:54.157710 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19015bd1-6e29-460c-92cd-b968030f477c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:54.162592 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbce43-f75c-42d5-bea7-abb2cda6ad46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:54.163276 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbce43-f75c-42d5-bea7-abb2cda6ad46 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:54.163562 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbce43-f75c-42d5-bea7-abb2cda6ad46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:54.163841 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbce43-f75c-42d5-bea7-abb2cda6ad46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:54.164600 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bbce43-f75c-42d5-bea7-abb2cda6ad46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:54.164916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1041/2065] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:54.168872 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bbce43-f75c-42d5-bea7-abb2cda6ad46 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:54.172517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e62fe4f-dfd3-4f50-878f-ca54fdd69b44 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:54.181059 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1025/2066] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:34:54 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:34:54.185258 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e62fe4f-dfd3-4f50-878f-ca54fdd69b44 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:54.185481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbce43-f75c-42d5-bea7-abb2cda6ad46 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:54.192666 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c007126-40e6-4c52-a4da-20111f082fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:54.193446 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c007126-40e6-4c52-a4da-20111f082fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:54.193610 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c007126-40e6-4c52-a4da-20111f082fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:54.194179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c007126-40e6-4c52-a4da-20111f082fb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:54.194976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c007126-40e6-4c52-a4da-20111f082fb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:34:54.195291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c007126-40e6-4c52-a4da-20111f082fb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:54.197030 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbce43-f75c-42d5-bea7-abb2cda6ad46 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:54.197519 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c007126-40e6-4c52-a4da-20111f082fb8 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:54.206454 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834efc4-8cef-4758-96b4-d0a634144b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:54.206737 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834efc4-8cef-4758-96b4-d0a634144b9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:54.206835 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834efc4-8cef-4758-96b4-d0a634144b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:54.207171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834efc4-8cef-4758-96b4-d0a634144b9f admin admin] PATH_INFO: `/v3/projects/b5d35bf9fba545168762f4714ed2cac2/users/73c7ca3c28a14b7e8e59d9b3a1d806b9/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:54.212248 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1042/2067] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:34:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:34:54.217759 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c007126-40e6-4c52-a4da-20111f082fb8 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:54.229526 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c007126-40e6-4c52-a4da-20111f082fb8 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:54.230895 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4834efc4-8cef-4758-96b4-d0a634144b9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5d35bf9fba545168762f4714ed2cac2, user_id=73c7ca3c28a14b7e8e59d9b3a1d806b9, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:54.233663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4834efc4-8cef-4758-96b4-d0a634144b9f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:54.235165 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e46a85-a897-4fb6-a042-6971dd2a1f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:54.235683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e46a85-a897-4fb6-a042-6971dd2a1f48 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:54.236022 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e46a85-a897-4fb6-a042-6971dd2a1f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:54.236625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e46a85-a897-4fb6-a042-6971dd2a1f48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:54.236980 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e46a85-a897-4fb6-a042-6971dd2a1f48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:54.239260 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e46a85-a897-4fb6-a042-6971dd2a1f48 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:54.254469 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1026/2068] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:34:54 2022] PUT /identity/v3/projects/b5d35bf9fba545168762f4714ed2cac2/users/73c7ca3c28a14b7e8e59d9b3a1d806b9/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:34:54.270965 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13244acb-730b-4a9a-a0c4-6ea4899ac683 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:54.271261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13244acb-730b-4a9a-a0c4-6ea4899ac683 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:54.271550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13244acb-730b-4a9a-a0c4-6ea4899ac683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:54.281304 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1043/2069] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:34:54 2022] POST /identity/v3/projects => generated 378 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:34:54.282368 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e46a85-a897-4fb6-a042-6971dd2a1f48 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:54.303004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:54.303664 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-13244acb-730b-4a9a-a0c4-6ea4899ac683 None None] MFA Rules not processed for user `73c7ca3c28a14b7e8e59d9b3a1d806b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:54.305264 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-13244acb-730b-4a9a-a0c4-6ea4899ac683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:54.349244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:54.365007 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e46a85-a897-4fb6-a042-6971dd2a1f48 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:54.369831 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54438371-17b8-4cdd-9870-9e677fe6187f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:54.370464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54438371-17b8-4cdd-9870-9e677fe6187f admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:54.370798 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54438371-17b8-4cdd-9870-9e677fe6187f admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:54.371019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54438371-17b8-4cdd-9870-9e677fe6187f admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:54.371751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54438371-17b8-4cdd-9870-9e677fe6187f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:54.373953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54438371-17b8-4cdd-9870-9e677fe6187f admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:54.398118 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1044/2070] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:34:54 2022] POST /identity/v3/users => generated 290 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:34:54.400137 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1027/2071] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:34:54.411610 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54438371-17b8-4cdd-9870-9e677fe6187f admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:54.413311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13244acb-730b-4a9a-a0c4-6ea4899ac683 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:54.416939 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-13244acb-730b-4a9a-a0c4-6ea4899ac683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:54.423124 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54438371-17b8-4cdd-9870-9e677fe6187f admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:54.430616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e325d6-6541-41cb-95ec-dcb3ff870888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:54.431898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e325d6-6541-41cb-95ec-dcb3ff870888 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:54.432239 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e325d6-6541-41cb-95ec-dcb3ff870888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:54.432552 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e325d6-6541-41cb-95ec-dcb3ff870888 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:54.433412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e325d6-6541-41cb-95ec-dcb3ff870888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:54.435173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e325d6-6541-41cb-95ec-dcb3ff870888 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:54.446839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1045/2072] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:34:54 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:34:54.451270 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e325d6-6541-41cb-95ec-dcb3ff870888 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:54.461816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e325d6-6541-41cb-95ec-dcb3ff870888 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:54.467653 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a712cc-7aae-48c1-bf8d-c1a9c8a0cb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:54.468718 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a712cc-7aae-48c1-bf8d-c1a9c8a0cb16 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:54.469008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a712cc-7aae-48c1-bf8d-c1a9c8a0cb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:54.469292 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a712cc-7aae-48c1-bf8d-c1a9c8a0cb16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:54.470016 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a712cc-7aae-48c1-bf8d-c1a9c8a0cb16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:54.471826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a712cc-7aae-48c1-bf8d-c1a9c8a0cb16 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:54.483290 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13244acb-730b-4a9a-a0c4-6ea4899ac683 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:54.487724 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ccab41-7e96-490b-90b8-0455baf5cbad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:54.487977 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1046/2073] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:34:54 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:34:54.488487 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccab41-7e96-490b-90b8-0455baf5cbad service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:54.488867 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccab41-7e96-490b-90b8-0455baf5cbad service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:54.488977 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a712cc-7aae-48c1-bf8d-c1a9c8a0cb16 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:54.489215 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccab41-7e96-490b-90b8-0455baf5cbad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:54.498754 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ccab41-7e96-490b-90b8-0455baf5cbad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:54.500346 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ccab41-7e96-490b-90b8-0455baf5cbad service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:54.500933 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a712cc-7aae-48c1-bf8d-c1a9c8a0cb16 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:54.506691 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb950a80-1a33-4d62-8389-15d82a726452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:54.507678 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb950a80-1a33-4d62-8389-15d82a726452 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:54.508002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb950a80-1a33-4d62-8389-15d82a726452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:54.508339 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb950a80-1a33-4d62-8389-15d82a726452 admin admin] PATH_INFO: `/v3/projects/8c04f4b30cbe453791f6edac8743ba29/users/633161799f924913bfa46ff24db44524/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:54.511113 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1028/2074] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:34:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:34:54.528945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb950a80-1a33-4d62-8389-15d82a726452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c04f4b30cbe453791f6edac8743ba29, user_id=633161799f924913bfa46ff24db44524, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:54.532618 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb950a80-1a33-4d62-8389-15d82a726452 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:54.556347 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1047/2075] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:34:54 2022] PUT /identity/v3/projects/8c04f4b30cbe453791f6edac8743ba29/users/633161799f924913bfa46ff24db44524/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:54.572637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1de21-afff-4925-9c62-de02266656f1 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:54.573048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1de21-afff-4925-9c62-de02266656f1 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:54.573545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1de21-afff-4925-9c62-de02266656f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:54.613378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-3da1de21-afff-4925-9c62-de02266656f1 None None] MFA Rules not processed for user `633161799f924913bfa46ff24db44524`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:54.613378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3da1de21-afff-4925-9c62-de02266656f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:54.714639 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1029/2076] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:54.715852 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb950a80-1a33-4d62-8389-15d82a726452 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:54.816109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb950a80-1a33-4d62-8389-15d82a726452 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:54.821991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f1de2-4b69-44ba-be96-e70783a5bb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:54.822743 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f1de2-4b69-44ba-be96-e70783a5bb32 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:54.823359 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f1de2-4b69-44ba-be96-e70783a5bb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:54.823655 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f1de2-4b69-44ba-be96-e70783a5bb32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:54.824708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f1de2-4b69-44ba-be96-e70783a5bb32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:54.827146 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f1de2-4b69-44ba-be96-e70783a5bb32 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:54.861308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1048/2077] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:34:54 2022] POST /identity/v3/projects => generated 378 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:34:54.865852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1de21-afff-4925-9c62-de02266656f1 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:54.879635 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1de21-afff-4925-9c62-de02266656f1 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:54.963249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21434de7-e95c-4dd5-9f69-6f62c088fb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:54.963902 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21434de7-e95c-4dd5-9f69-6f62c088fb12 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:54.964244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21434de7-e95c-4dd5-9f69-6f62c088fb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:54.964489 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21434de7-e95c-4dd5-9f69-6f62c088fb12 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:54.965176 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21434de7-e95c-4dd5-9f69-6f62c088fb12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:54.967123 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21434de7-e95c-4dd5-9f69-6f62c088fb12 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:54.987234 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:54.994584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1030/2078] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:34:54 2022] POST /identity/v3/users => generated 290 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:34:54.998459 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f1de2-4b69-44ba-be96-e70783a5bb32 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:55.010716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f1de2-4b69-44ba-be96-e70783a5bb32 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:55.016611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a9cda-66c4-476a-9a78-088ee378996b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:55.017693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a9cda-66c4-476a-9a78-088ee378996b admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:55.018076 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a9cda-66c4-476a-9a78-088ee378996b admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:55.018447 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a9cda-66c4-476a-9a78-088ee378996b admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:55.019421 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a9cda-66c4-476a-9a78-088ee378996b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:55.023355 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a9cda-66c4-476a-9a78-088ee378996b admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:55.034406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1049/2079] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:34:54 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:34:55.038823 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21434de7-e95c-4dd5-9f69-6f62c088fb12 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:55.049968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21434de7-e95c-4dd5-9f69-6f62c088fb12 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:55.055653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fccc29-904f-4fb5-be2e-628b497faad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:55.056500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fccc29-904f-4fb5-be2e-628b497faad5 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:55.056761 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fccc29-904f-4fb5-be2e-628b497faad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:55.057057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fccc29-904f-4fb5-be2e-628b497faad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:55.057817 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fccc29-904f-4fb5-be2e-628b497faad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:55.059486 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fccc29-904f-4fb5-be2e-628b497faad5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:55.073268 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1031/2080] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:34:55 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:34:55.079968 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a9cda-66c4-476a-9a78-088ee378996b admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:55.091372 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a9cda-66c4-476a-9a78-088ee378996b admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:55.096853 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06e14f-281c-4f4a-b224-56ff5980618d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:55.097601 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06e14f-281c-4f4a-b224-56ff5980618d admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:55.097884 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06e14f-281c-4f4a-b224-56ff5980618d admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:55.098165 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06e14f-281c-4f4a-b224-56ff5980618d admin admin] PATH_INFO: `/v3/projects/b074296506a94bbb9d29164fe68b62a9/users/f0c5c42846fc4d3495f2d452e547512d/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:55.116188 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06e14f-281c-4f4a-b224-56ff5980618d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b074296506a94bbb9d29164fe68b62a9, user_id=f0c5c42846fc4d3495f2d452e547512d, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:55.118997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06e14f-281c-4f4a-b224-56ff5980618d admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:55.137291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1050/2081] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:34:55 2022] PUT /identity/v3/projects/b074296506a94bbb9d29164fe68b62a9/users/f0c5c42846fc4d3495f2d452e547512d/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:55.150945 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d5139-d7f7-4fae-b84e-79abd3a253f3 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:55.151431 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d5139-d7f7-4fae-b84e-79abd3a253f3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:55.151905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d5139-d7f7-4fae-b84e-79abd3a253f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:55.185280 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d78d5139-d7f7-4fae-b84e-79abd3a253f3 None None] MFA Rules not processed for user `f0c5c42846fc4d3495f2d452e547512d`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:55.186558 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d78d5139-d7f7-4fae-b84e-79abd3a253f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:55.259829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1032/2082] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:55 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:55.267423 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06e14f-281c-4f4a-b224-56ff5980618d admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:55.278853 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:55.337281 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06e14f-281c-4f4a-b224-56ff5980618d admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:55.344166 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa48d5-856e-4f74-97dd-6e7022cfc265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:55.344829 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa48d5-856e-4f74-97dd-6e7022cfc265 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:55.345144 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa48d5-856e-4f74-97dd-6e7022cfc265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:55.345414 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa48d5-856e-4f74-97dd-6e7022cfc265 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:55.346433 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa48d5-856e-4f74-97dd-6e7022cfc265 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:55.348510 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa48d5-856e-4f74-97dd-6e7022cfc265 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:55.382593 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1051/2083] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:34:55 2022] POST /identity/v3/projects => generated 376 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:34:55.387546 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d5139-d7f7-4fae-b84e-79abd3a253f3 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:55.401162 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d5139-d7f7-4fae-b84e-79abd3a253f3 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:55.467414 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9024d0-a33f-42b5-823a-5fe802f36fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:55.468098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9024d0-a33f-42b5-823a-5fe802f36fbb admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:55.468413 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9024d0-a33f-42b5-823a-5fe802f36fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:55.468687 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9024d0-a33f-42b5-823a-5fe802f36fbb admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:55.469379 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9024d0-a33f-42b5-823a-5fe802f36fbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:55.471377 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9024d0-a33f-42b5-823a-5fe802f36fbb admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:55.496411 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1033/2084] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:34:55 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:34:55.501645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa48d5-856e-4f74-97dd-6e7022cfc265 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:55.513103 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa48d5-856e-4f74-97dd-6e7022cfc265 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:55.518964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f51ff-75ce-4c8d-82e3-c1b8f9d65530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:55.519609 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f51ff-75ce-4c8d-82e3-c1b8f9d65530 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:55.519895 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f51ff-75ce-4c8d-82e3-c1b8f9d65530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:55.520220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f51ff-75ce-4c8d-82e3-c1b8f9d65530 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:55.520933 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f51ff-75ce-4c8d-82e3-c1b8f9d65530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:55.522613 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f51ff-75ce-4c8d-82e3-c1b8f9d65530 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:55.537005 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1052/2085] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:34:55 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:34:55.544208 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9024d0-a33f-42b5-823a-5fe802f36fbb admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:55.557660 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9024d0-a33f-42b5-823a-5fe802f36fbb admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:55.566511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4814d-cef1-46bf-beff-09264e7e9796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:55.567428 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4814d-cef1-46bf-beff-09264e7e9796 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:55.567684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4814d-cef1-46bf-beff-09264e7e9796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:55.568156 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4814d-cef1-46bf-beff-09264e7e9796 admin admin] PATH_INFO: `/v3/projects/50b0a679c16f4dabb96ef1ca9b8c7688/users/1d30a050e2d147b6a74d175be5265173/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:55.588407 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4814d-cef1-46bf-beff-09264e7e9796 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50b0a679c16f4dabb96ef1ca9b8c7688, user_id=1d30a050e2d147b6a74d175be5265173, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:55.591200 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4814d-cef1-46bf-beff-09264e7e9796 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:55.616833 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1034/2086] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:34:55 2022] PUT /identity/v3/projects/50b0a679c16f4dabb96ef1ca9b8c7688/users/1d30a050e2d147b6a74d175be5265173/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:55.621507 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f51ff-75ce-4c8d-82e3-c1b8f9d65530 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:55.637852 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f51ff-75ce-4c8d-82e3-c1b8f9d65530 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:55.725424 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79377b79-2e4e-44ed-8267-186b5c732d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:55.726083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79377b79-2e4e-44ed-8267-186b5c732d10 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:55.726364 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79377b79-2e4e-44ed-8267-186b5c732d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:55.726638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79377b79-2e4e-44ed-8267-186b5c732d10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:55.727717 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79377b79-2e4e-44ed-8267-186b5c732d10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:55.729447 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79377b79-2e4e-44ed-8267-186b5c732d10 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:55.742737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1053/2087] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:34:55 2022] GET /identity/v3/roles => generated 1427 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:34:55.747536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4814d-cef1-46bf-beff-09264e7e9796 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:55.762098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4814d-cef1-46bf-beff-09264e7e9796 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:55.769324 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da58b2b-cf9d-4268-bd8d-e625a72501d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:55.770063 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da58b2b-cf9d-4268-bd8d-e625a72501d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:55.770331 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da58b2b-cf9d-4268-bd8d-e625a72501d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:55.770615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da58b2b-cf9d-4268-bd8d-e625a72501d3 admin admin] PATH_INFO: `/v3/projects/50b0a679c16f4dabb96ef1ca9b8c7688/users/1d30a050e2d147b6a74d175be5265173/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:55.774306 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da58b2b-cf9d-4268-bd8d-e625a72501d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50b0a679c16f4dabb96ef1ca9b8c7688, user_id=1d30a050e2d147b6a74d175be5265173, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:55.777208 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da58b2b-cf9d-4268-bd8d-e625a72501d3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:55.790609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1035/2088] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:34:55 2022] PUT /identity/v3/projects/50b0a679c16f4dabb96ef1ca9b8c7688/users/1d30a050e2d147b6a74d175be5265173/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:34:55.804551 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551b9f9-2cd7-4349-b622-74cb8a26b70d None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:55.804551 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551b9f9-2cd7-4349-b622-74cb8a26b70d None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:55.804551 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551b9f9-2cd7-4349-b622-74cb8a26b70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:55.850539 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-d551b9f9-2cd7-4349-b622-74cb8a26b70d None None] MFA Rules not processed for user `1d30a050e2d147b6a74d175be5265173`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:34:55.852949 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d551b9f9-2cd7-4349-b622-74cb8a26b70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:55.956451 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1054/2089] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:34:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:34:55.969785 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da58b2b-cf9d-4268-bd8d-e625a72501d3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:55.971983 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3da58b2b-cf9d-4268-bd8d-e625a72501d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:34:56.013694 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:56.048296 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da58b2b-cf9d-4268-bd8d-e625a72501d3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:56.052614 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960635d-2339-4290-af69-59c20041d3c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:56.054044 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960635d-2339-4290-af69-59c20041d3c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:56.054044 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960635d-2339-4290-af69-59c20041d3c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:56.054044 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960635d-2339-4290-af69-59c20041d3c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:56.133338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960635d-2339-4290-af69-59c20041d3c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:56.134942 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960635d-2339-4290-af69-59c20041d3c9 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:56.146686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1036/2090] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:34:55 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:58.951526 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551b9f9-2cd7-4349-b622-74cb8a26b70d None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:58.958094 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:34:58.963428 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551b9f9-2cd7-4349-b622-74cb8a26b70d None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:58.969600 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f47c0-fe3f-4127-a648-886f2e04f9a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:58.970332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f47c0-fe3f-4127-a648-886f2e04f9a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:58.971134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f47c0-fe3f-4127-a648-886f2e04f9a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:58.971691 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f47c0-fe3f-4127-a648-886f2e04f9a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:59.040509 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f47c0-fe3f-4127-a648-886f2e04f9a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:59.042300 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f47c0-fe3f-4127-a648-886f2e04f9a8 service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:59.053036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1055/2091] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:34:58 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:34:59.780336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960635d-2339-4290-af69-59c20041d3c9 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:34:59.790607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960635d-2339-4290-af69-59c20041d3c9 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:34:59.794883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f388e4-530b-4bef-8fde-5c4489ea9a24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:34:59.795524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f388e4-530b-4bef-8fde-5c4489ea9a24 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:34:59.795811 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f388e4-530b-4bef-8fde-5c4489ea9a24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:34:59.796108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f388e4-530b-4bef-8fde-5c4489ea9a24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:34:59.809000 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f388e4-530b-4bef-8fde-5c4489ea9a24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:34:59.810696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f388e4-530b-4bef-8fde-5c4489ea9a24 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:34:59.823175 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1037/2092] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:34:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:35:07.516570 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f47c0-fe3f-4127-a648-886f2e04f9a8 service cinder] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:07.621394 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f47c0-fe3f-4127-a648-886f2e04f9a8 service cinder] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:07.626797 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028c1b5-d7b8-4e4e-94b6-af7bf3236846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:07.627679 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028c1b5-d7b8-4e4e-94b6-af7bf3236846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:07.627987 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028c1b5-d7b8-4e4e-94b6-af7bf3236846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:07.628277 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028c1b5-d7b8-4e4e-94b6-af7bf3236846 admin admin] PATH_INFO: `/v3/users/633161799f924913bfa46ff24db44524` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:07.629647 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028c1b5-d7b8-4e4e-94b6-af7bf3236846 admin admin] RBAC: Authorizing `identity:delete_user(user_id=633161799f924913bfa46ff24db44524)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:07.633239 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028c1b5-d7b8-4e4e-94b6-af7bf3236846 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:07.702249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-3028c1b5-d7b8-4e4e-94b6-af7bf3236846 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '633161799f924913bfa46ff24db44524'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:07.703033 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-3028c1b5-d7b8-4e4e-94b6-af7bf3236846 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '633161799f924913bfa46ff24db44524'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:07.724904 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-3028c1b5-d7b8-4e4e-94b6-af7bf3236846 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '633161799f924913bfa46ff24db44524'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:07.734997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-3028c1b5-d7b8-4e4e-94b6-af7bf3236846 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '633161799f924913bfa46ff24db44524'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:07.743866 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-3028c1b5-d7b8-4e4e-94b6-af7bf3236846 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '633161799f924913bfa46ff24db44524'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:07.750248 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1056/2093] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:35:07 2022] DELETE /identity/v3/users/633161799f924913bfa46ff24db44524 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:35:07.755056 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc5ebe-e1ce-498e-90b4-ae1c5b8d7678 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:07.755609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc5ebe-e1ce-498e-90b4-ae1c5b8d7678 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:07.755609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc5ebe-e1ce-498e-90b4-ae1c5b8d7678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:07.776611 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8afc5ebe-e1ce-498e-90b4-ae1c5b8d7678 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:07.777861 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8afc5ebe-e1ce-498e-90b4-ae1c5b8d7678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:07.841788 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1038/2094] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:35:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:35:07.852653 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028c1b5-d7b8-4e4e-94b6-af7bf3236846 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:07.854177 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-3028c1b5-d7b8-4e4e-94b6-af7bf3236846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:07.883680 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:07.918968 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028c1b5-d7b8-4e4e-94b6-af7bf3236846 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:07.923348 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b92f57-a87b-4ff6-a036-1ec101d2bb43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:07.924021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b92f57-a87b-4ff6-a036-1ec101d2bb43 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:07.924305 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b92f57-a87b-4ff6-a036-1ec101d2bb43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:07.924581 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b92f57-a87b-4ff6-a036-1ec101d2bb43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:07.934618 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b92f57-a87b-4ff6-a036-1ec101d2bb43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:07.936137 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b92f57-a87b-4ff6-a036-1ec101d2bb43 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:07.946846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1057/2095] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:35:07 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:35:08.324561 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc5ebe-e1ce-498e-90b4-ae1c5b8d7678 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:08.326154 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8afc5ebe-e1ce-498e-90b4-ae1c5b8d7678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:08.346717 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc5ebe-e1ce-498e-90b4-ae1c5b8d7678 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:08.352988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3c2b0-24c5-4d91-81ad-23d3934f75a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:08.353836 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3c2b0-24c5-4d91-81ad-23d3934f75a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:08.354138 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3c2b0-24c5-4d91-81ad-23d3934f75a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:08.354411 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3c2b0-24c5-4d91-81ad-23d3934f75a5 admin admin] PATH_INFO: `/v3/projects/8c04f4b30cbe453791f6edac8743ba29` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:08.355843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3c2b0-24c5-4d91-81ad-23d3934f75a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c04f4b30cbe453791f6edac8743ba29)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:08.357875 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3c2b0-24c5-4d91-81ad-23d3934f75a5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:08.413923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-40b3c2b0-24c5-4d91-81ad-23d3934f75a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c04f4b30cbe453791f6edac8743ba29', 'request_id': 'req-40b3c2b0-24c5-4d91-81ad-23d3934f75a5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:08.421889 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-40b3c2b0-24c5-4d91-81ad-23d3934f75a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c04f4b30cbe453791f6edac8743ba29', 'request_id': 'req-40b3c2b0-24c5-4d91-81ad-23d3934f75a5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:08.428960 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-40b3c2b0-24c5-4d91-81ad-23d3934f75a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c04f4b30cbe453791f6edac8743ba29', 'request_id': 'req-40b3c2b0-24c5-4d91-81ad-23d3934f75a5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:08.432062 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:08.447592 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-40b3c2b0-24c5-4d91-81ad-23d3934f75a5 admin admin] The token cache is being invalidate because project 8c04f4b30cbe453791f6edac8743ba29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:35:08.448027 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-40b3c2b0-24c5-4d91-81ad-23d3934f75a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:08.448659 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-40b3c2b0-24c5-4d91-81ad-23d3934f75a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:08.450255 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1039/2096] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:35:08 2022] DELETE /identity/v3/projects/8c04f4b30cbe453791f6edac8743ba29 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:35:08.454841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b92f57-a87b-4ff6-a036-1ec101d2bb43 service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:08.466356 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b92f57-a87b-4ff6-a036-1ec101d2bb43 service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:08.546806 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143b72f-d432-42b3-824d-2555816389f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:08.547706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143b72f-d432-42b3-824d-2555816389f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:08.548227 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143b72f-d432-42b3-824d-2555816389f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:08.548653 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143b72f-d432-42b3-824d-2555816389f2 admin admin] PATH_INFO: `/v3/users/f0c5c42846fc4d3495f2d452e547512d` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:08.550040 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143b72f-d432-42b3-824d-2555816389f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0c5c42846fc4d3495f2d452e547512d)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:08.552840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143b72f-d432-42b3-824d-2555816389f2 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:08.611756 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f143b72f-d432-42b3-824d-2555816389f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0c5c42846fc4d3495f2d452e547512d'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:08.612682 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f143b72f-d432-42b3-824d-2555816389f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0c5c42846fc4d3495f2d452e547512d'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:08.627168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f143b72f-d432-42b3-824d-2555816389f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0c5c42846fc4d3495f2d452e547512d'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:08.635053 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f143b72f-d432-42b3-824d-2555816389f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0c5c42846fc4d3495f2d452e547512d'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:08.644953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f143b72f-d432-42b3-824d-2555816389f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0c5c42846fc4d3495f2d452e547512d'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:08.653047 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1058/2097] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:35:08 2022] DELETE /identity/v3/users/f0c5c42846fc4d3495f2d452e547512d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:35:08.986779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3c2b0-24c5-4d91-81ad-23d3934f75a5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:08.988864 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-40b3c2b0-24c5-4d91-81ad-23d3934f75a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:09.007028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3c2b0-24c5-4d91-81ad-23d3934f75a5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:09.087765 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab88f85-056e-4eb4-af42-865e96e72fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:09.088384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab88f85-056e-4eb4-af42-865e96e72fd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:09.088664 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab88f85-056e-4eb4-af42-865e96e72fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:09.088976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab88f85-056e-4eb4-af42-865e96e72fd7 admin admin] PATH_INFO: `/v3/projects/b074296506a94bbb9d29164fe68b62a9` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:09.090154 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab88f85-056e-4eb4-af42-865e96e72fd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b074296506a94bbb9d29164fe68b62a9)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:09.092159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab88f85-056e-4eb4-af42-865e96e72fd7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:09.170557 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9ab88f85-056e-4eb4-af42-865e96e72fd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b074296506a94bbb9d29164fe68b62a9', 'request_id': 'req-9ab88f85-056e-4eb4-af42-865e96e72fd7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:09.182717 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9ab88f85-056e-4eb4-af42-865e96e72fd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b074296506a94bbb9d29164fe68b62a9', 'request_id': 'req-9ab88f85-056e-4eb4-af42-865e96e72fd7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:09.191283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9ab88f85-056e-4eb4-af42-865e96e72fd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b074296506a94bbb9d29164fe68b62a9', 'request_id': 'req-9ab88f85-056e-4eb4-af42-865e96e72fd7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:09.206379 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9ab88f85-056e-4eb4-af42-865e96e72fd7 admin admin] The token cache is being invalidate because project b074296506a94bbb9d29164fe68b62a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:35:09.206730 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9ab88f85-056e-4eb4-af42-865e96e72fd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:09.207400 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9ab88f85-056e-4eb4-af42-865e96e72fd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:09.209179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1040/2098] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:35:08 2022] DELETE /identity/v3/projects/b074296506a94bbb9d29164fe68b62a9 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:35:09.215482 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143b72f-d432-42b3-824d-2555816389f2 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:09.217457 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-f143b72f-d432-42b3-824d-2555816389f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:09.231852 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143b72f-d432-42b3-824d-2555816389f2 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:09.284543 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:09.304348 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d0423-b0be-4984-b896-0baa81f4d965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:09.305080 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d0423-b0be-4984-b896-0baa81f4d965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:09.305385 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d0423-b0be-4984-b896-0baa81f4d965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:09.305621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d0423-b0be-4984-b896-0baa81f4d965 admin admin] PATH_INFO: `/v3/users/1d30a050e2d147b6a74d175be5265173` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:09.306992 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d0423-b0be-4984-b896-0baa81f4d965 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d30a050e2d147b6a74d175be5265173)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:09.309167 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d0423-b0be-4984-b896-0baa81f4d965 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:09.367998 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-ba4d0423-b0be-4984-b896-0baa81f4d965 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d30a050e2d147b6a74d175be5265173'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:09.368575 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-ba4d0423-b0be-4984-b896-0baa81f4d965 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d30a050e2d147b6a74d175be5265173'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:09.381918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-ba4d0423-b0be-4984-b896-0baa81f4d965 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d30a050e2d147b6a74d175be5265173'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:09.394461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-ba4d0423-b0be-4984-b896-0baa81f4d965 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d30a050e2d147b6a74d175be5265173'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:09.404589 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-ba4d0423-b0be-4984-b896-0baa81f4d965 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d30a050e2d147b6a74d175be5265173'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:09.412234 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1059/2099] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:35:09 2022] DELETE /identity/v3/users/1d30a050e2d147b6a74d175be5265173 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:35:09.763108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab88f85-056e-4eb4-af42-865e96e72fd7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:09.764720 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9ab88f85-056e-4eb4-af42-865e96e72fd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:09.779106 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab88f85-056e-4eb4-af42-865e96e72fd7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:09.856544 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b88eb02-ab22-461b-a10b-61218766ec9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:09.857314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88eb02-ab22-461b-a10b-61218766ec9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:09.857681 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88eb02-ab22-461b-a10b-61218766ec9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:09.858038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88eb02-ab22-461b-a10b-61218766ec9b admin admin] PATH_INFO: `/v3/projects/50b0a679c16f4dabb96ef1ca9b8c7688` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:09.859437 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b88eb02-ab22-461b-a10b-61218766ec9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=50b0a679c16f4dabb96ef1ca9b8c7688)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:09.861580 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b88eb02-ab22-461b-a10b-61218766ec9b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:09.903169 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:09.929551 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1b88eb02-ab22-461b-a10b-61218766ec9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50b0a679c16f4dabb96ef1ca9b8c7688', 'request_id': 'req-1b88eb02-ab22-461b-a10b-61218766ec9b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:09.937280 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1b88eb02-ab22-461b-a10b-61218766ec9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50b0a679c16f4dabb96ef1ca9b8c7688', 'request_id': 'req-1b88eb02-ab22-461b-a10b-61218766ec9b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:09.945428 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1b88eb02-ab22-461b-a10b-61218766ec9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50b0a679c16f4dabb96ef1ca9b8c7688', 'request_id': 'req-1b88eb02-ab22-461b-a10b-61218766ec9b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:09.960257 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1b88eb02-ab22-461b-a10b-61218766ec9b admin admin] The token cache is being invalidate because project 50b0a679c16f4dabb96ef1ca9b8c7688 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:35:09.960536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1b88eb02-ab22-461b-a10b-61218766ec9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:09.961258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1b88eb02-ab22-461b-a10b-61218766ec9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:09.962912 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1041/2100] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:35:09 2022] DELETE /identity/v3/projects/50b0a679c16f4dabb96ef1ca9b8c7688 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:35:09.968252 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcaae5-14c2-4fec-a0c8-b1e277b601df None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:09.968828 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcaae5-14c2-4fec-a0c8-b1e277b601df None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:09.968828 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcaae5-14c2-4fec-a0c8-b1e277b601df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:09.993316 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-84bcaae5-14c2-4fec-a0c8-b1e277b601df None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:09.994828 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-84bcaae5-14c2-4fec-a0c8-b1e277b601df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:10.082666 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1060/2101] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:35:09 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:35:10.092052 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a6609-2bda-4cce-8cec-db9050ac478a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:10.092750 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a6609-2bda-4cce-8cec-db9050ac478a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:10.092750 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a6609-2bda-4cce-8cec-db9050ac478a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:10.117772 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a45a6609-2bda-4cce-8cec-db9050ac478a None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:10.119195 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a45a6609-2bda-4cce-8cec-db9050ac478a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:10.128278 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1042/2102] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:35:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:35:10.133127 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcaae5-14c2-4fec-a0c8-b1e277b601df None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:10.145851 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcaae5-14c2-4fec-a0c8-b1e277b601df None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:10.152289 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250163d-18fe-4a23-9101-4c11d9859d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:10.152930 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250163d-18fe-4a23-9101-4c11d9859d42 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:10.153211 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250163d-18fe-4a23-9101-4c11d9859d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:10.153484 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250163d-18fe-4a23-9101-4c11d9859d42 admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:10.154242 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e250163d-18fe-4a23-9101-4c11d9859d42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:35:10.154549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e250163d-18fe-4a23-9101-4c11d9859d42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:10.156258 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e250163d-18fe-4a23-9101-4c11d9859d42 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:10.170514 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1061/2103] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:35:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:35:10.177703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a6609-2bda-4cce-8cec-db9050ac478a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:10.191686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a6609-2bda-4cce-8cec-db9050ac478a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:10.200934 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0172c6-9f06-45a5-afeb-1a81a4612e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:10.202071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0172c6-9f06-45a5-afeb-1a81a4612e9b admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:10.202617 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0172c6-9f06-45a5-afeb-1a81a4612e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:10.203159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0172c6-9f06-45a5-afeb-1a81a4612e9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:10.204676 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0172c6-9f06-45a5-afeb-1a81a4612e9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:10.208960 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0172c6-9f06-45a5-afeb-1a81a4612e9b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:10.264029 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1043/2104] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:35:10 2022] POST /identity/v3/projects => generated 374 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:35:10.269098 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250163d-18fe-4a23-9101-4c11d9859d42 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:10.282095 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250163d-18fe-4a23-9101-4c11d9859d42 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:10.348392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:10.369396 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415429ea-dce3-43ff-a9e5-d4a35de1890d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:10.370578 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415429ea-dce3-43ff-a9e5-d4a35de1890d admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:10.371062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415429ea-dce3-43ff-a9e5-d4a35de1890d admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:10.372250 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415429ea-dce3-43ff-a9e5-d4a35de1890d admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:10.373123 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415429ea-dce3-43ff-a9e5-d4a35de1890d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:10.377456 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415429ea-dce3-43ff-a9e5-d4a35de1890d admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:10.414274 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1062/2105] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:35:10 2022] POST /identity/v3/users => generated 288 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:35:10.419285 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0172c6-9f06-45a5-afeb-1a81a4612e9b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:10.434497 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0172c6-9f06-45a5-afeb-1a81a4612e9b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:10.441056 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b0b65-890d-44ae-97e3-96619563a9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:10.441835 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b0b65-890d-44ae-97e3-96619563a9f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:10.442231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b0b65-890d-44ae-97e3-96619563a9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:10.442562 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b0b65-890d-44ae-97e3-96619563a9f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:10.443522 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b0b65-890d-44ae-97e3-96619563a9f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:10.445465 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b0b65-890d-44ae-97e3-96619563a9f0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:10.459634 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1044/2106] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:35:10 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:35:10.465080 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415429ea-dce3-43ff-a9e5-d4a35de1890d admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:10.477587 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415429ea-dce3-43ff-a9e5-d4a35de1890d admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:10.485317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc92d4fb-7bbe-469f-812a-1b4be8dbc9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:10.486002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92d4fb-7bbe-469f-812a-1b4be8dbc9b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:10.486287 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92d4fb-7bbe-469f-812a-1b4be8dbc9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:10.486615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92d4fb-7bbe-469f-812a-1b4be8dbc9b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:10.487286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc92d4fb-7bbe-469f-812a-1b4be8dbc9b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:10.489053 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc92d4fb-7bbe-469f-812a-1b4be8dbc9b6 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:10.503363 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1063/2107] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:35:10 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:35:10.507336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b0b65-890d-44ae-97e3-96619563a9f0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:10.519158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b0b65-890d-44ae-97e3-96619563a9f0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:10.524970 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f044d1d1-8323-4c0c-95f6-f14ecfcc58d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:10.525769 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f044d1d1-8323-4c0c-95f6-f14ecfcc58d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:10.526068 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f044d1d1-8323-4c0c-95f6-f14ecfcc58d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:10.526361 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f044d1d1-8323-4c0c-95f6-f14ecfcc58d9 admin admin] PATH_INFO: `/v3/projects/99b1be624402454ba326a7d7a8754a25/users/a025ea88db864b55b7537311d20f4808/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:10.550344 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f044d1d1-8323-4c0c-95f6-f14ecfcc58d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99b1be624402454ba326a7d7a8754a25, user_id=a025ea88db864b55b7537311d20f4808, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:10.553279 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f044d1d1-8323-4c0c-95f6-f14ecfcc58d9 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:10.575597 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1045/2108] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:35:10 2022] PUT /identity/v3/projects/99b1be624402454ba326a7d7a8754a25/users/a025ea88db864b55b7537311d20f4808/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:35:10.583677 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a93e0-ef7a-452e-bc89-aac7317de62f None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:10.584096 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a93e0-ef7a-452e-bc89-aac7317de62f None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:10.584320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a93e0-ef7a-452e-bc89-aac7317de62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:10.610133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-a06a93e0-ef7a-452e-bc89-aac7317de62f None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:10.611533 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-a06a93e0-ef7a-452e-bc89-aac7317de62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:10.705023 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1064/2109] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:35:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:35:10.715835 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f044d1d1-8323-4c0c-95f6-f14ecfcc58d9 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:10.720095 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f044d1d1-8323-4c0c-95f6-f14ecfcc58d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:10.808787 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f044d1d1-8323-4c0c-95f6-f14ecfcc58d9 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:10.813074 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76870f-a950-4312-946b-1fb4a1278508 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:10.813687 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76870f-a950-4312-946b-1fb4a1278508 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:10.814005 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76870f-a950-4312-946b-1fb4a1278508 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:10.814282 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76870f-a950-4312-946b-1fb4a1278508 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:10.825960 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c76870f-a950-4312-946b-1fb4a1278508 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:10.827414 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c76870f-a950-4312-946b-1fb4a1278508 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:10.838760 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1046/2110] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:35:10 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:35:11.664480 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebee4c-730d-453a-92ed-c27cf7f4915f None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:11.665629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebee4c-730d-453a-92ed-c27cf7f4915f None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:11.665629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebee4c-730d-453a-92ed-c27cf7f4915f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:11.707616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-69ebee4c-730d-453a-92ed-c27cf7f4915f None None] MFA Rules not processed for user `a025ea88db864b55b7537311d20f4808`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:11.709208 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-69ebee4c-730d-453a-92ed-c27cf7f4915f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:11.808010 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1065/2111] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:35:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:35:11.835025 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76870f-a950-4312-946b-1fb4a1278508 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:11.838467 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4c76870f-a950-4312-946b-1fb4a1278508 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:11.929228 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76870f-a950-4312-946b-1fb4a1278508 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:11.933319 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3884a-ff2f-4dcf-b570-169083d19a55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:11.933977 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3884a-ff2f-4dcf-b570-169083d19a55 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:11.934261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3884a-ff2f-4dcf-b570-169083d19a55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:11.934539 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3884a-ff2f-4dcf-b570-169083d19a55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:11.952364 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:11.954476 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3884a-ff2f-4dcf-b570-169083d19a55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:11.958731 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3884a-ff2f-4dcf-b570-169083d19a55 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:11.978978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1047/2112] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:35:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:35:20.867091 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0105a3-8daa-4b20-82f0-838d21ab4c19 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:20.867736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0105a3-8daa-4b20-82f0-838d21ab4c19 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:20.867736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0105a3-8daa-4b20-82f0-838d21ab4c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:20.890187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-2f0105a3-8daa-4b20-82f0-838d21ab4c19 None None] MFA Rules not processed for user `a025ea88db864b55b7537311d20f4808`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:20.891518 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-2f0105a3-8daa-4b20-82f0-838d21ab4c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:20.899567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1066/2113] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:35:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:35:21.984845 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:25.826583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3884a-ff2f-4dcf-b570-169083d19a55 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:25.842612 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-74c3884a-ff2f-4dcf-b570-169083d19a55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:25.879567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3884a-ff2f-4dcf-b570-169083d19a55 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:25.893006 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed196b-2956-4eff-a056-ebdcd9d5b9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:25.893813 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed196b-2956-4eff-a056-ebdcd9d5b9f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:25.894207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed196b-2956-4eff-a056-ebdcd9d5b9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:25.894863 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed196b-2956-4eff-a056-ebdcd9d5b9f6 admin admin] PATH_INFO: `/v3/users/9520a49687b24281b78f08d43cb27dc3` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:25.901587 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed196b-2956-4eff-a056-ebdcd9d5b9f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9520a49687b24281b78f08d43cb27dc3)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:25.903064 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed196b-2956-4eff-a056-ebdcd9d5b9f6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:26.010360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-55ed196b-2956-4eff-a056-ebdcd9d5b9f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9520a49687b24281b78f08d43cb27dc3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:26.019999 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-55ed196b-2956-4eff-a056-ebdcd9d5b9f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9520a49687b24281b78f08d43cb27dc3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:26.020942 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-55ed196b-2956-4eff-a056-ebdcd9d5b9f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9520a49687b24281b78f08d43cb27dc3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:26.038706 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-55ed196b-2956-4eff-a056-ebdcd9d5b9f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9520a49687b24281b78f08d43cb27dc3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:26.039595 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-55ed196b-2956-4eff-a056-ebdcd9d5b9f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9520a49687b24281b78f08d43cb27dc3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:26.061811 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1048/2114] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:35:25 2022] DELETE /identity/v3/users/9520a49687b24281b78f08d43cb27dc3 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:35:26.311059 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0105a3-8daa-4b20-82f0-838d21ab4c19 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:26.323389 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0105a3-8daa-4b20-82f0-838d21ab4c19 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:26.378929 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:26.390083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75589c5-154a-4077-a575-154860852399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:26.390751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75589c5-154a-4077-a575-154860852399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:26.391092 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75589c5-154a-4077-a575-154860852399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:26.391375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75589c5-154a-4077-a575-154860852399 admin admin] PATH_INFO: `/v3/projects/c167faad577d47dfaf0d010a3f983fe3` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:26.392626 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75589c5-154a-4077-a575-154860852399 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c167faad577d47dfaf0d010a3f983fe3)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:26.394651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75589c5-154a-4077-a575-154860852399 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:26.459270 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f75589c5-154a-4077-a575-154860852399 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c167faad577d47dfaf0d010a3f983fe3', 'request_id': 'req-f75589c5-154a-4077-a575-154860852399'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:26.468565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f75589c5-154a-4077-a575-154860852399 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c167faad577d47dfaf0d010a3f983fe3', 'request_id': 'req-f75589c5-154a-4077-a575-154860852399'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:26.475852 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f75589c5-154a-4077-a575-154860852399 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c167faad577d47dfaf0d010a3f983fe3', 'request_id': 'req-f75589c5-154a-4077-a575-154860852399'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:26.488136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f75589c5-154a-4077-a575-154860852399 admin admin] The token cache is being invalidate because project c167faad577d47dfaf0d010a3f983fe3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:35:26.488443 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f75589c5-154a-4077-a575-154860852399 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:26.489227 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f75589c5-154a-4077-a575-154860852399 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:26.491214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1067/2115] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:35:26 2022] DELETE /identity/v3/projects/c167faad577d47dfaf0d010a3f983fe3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:35:26.496724 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf03826-de93-4edb-9f93-81284e44ff5a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:26.497249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf03826-de93-4edb-9f93-81284e44ff5a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:26.497529 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf03826-de93-4edb-9f93-81284e44ff5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:26.520164 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-edf03826-de93-4edb-9f93-81284e44ff5a None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:26.521438 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-edf03826-de93-4edb-9f93-81284e44ff5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:26.586845 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1049/2116] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:35:26 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:35:26.598839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b17e9-1795-43da-8699-5c854a7f5e0d None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:26.599172 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b17e9-1795-43da-8699-5c854a7f5e0d None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:26.599406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b17e9-1795-43da-8699-5c854a7f5e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:26.620341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-ce6b17e9-1795-43da-8699-5c854a7f5e0d None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:26.622021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-ce6b17e9-1795-43da-8699-5c854a7f5e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:26.630936 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1068/2117] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:35:26 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:35:26.635577 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf03826-de93-4edb-9f93-81284e44ff5a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:26.646637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf03826-de93-4edb-9f93-81284e44ff5a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:26.652278 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554af01f-7df2-4b00-8219-87a5e5b0fe39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:26.652932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554af01f-7df2-4b00-8219-87a5e5b0fe39 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:26.653219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554af01f-7df2-4b00-8219-87a5e5b0fe39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:26.653496 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554af01f-7df2-4b00-8219-87a5e5b0fe39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:26.654221 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554af01f-7df2-4b00-8219-87a5e5b0fe39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:35:26.654535 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554af01f-7df2-4b00-8219-87a5e5b0fe39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:26.656217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554af01f-7df2-4b00-8219-87a5e5b0fe39 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:26.669531 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1050/2118] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:35:26 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:35:26.675271 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b17e9-1795-43da-8699-5c854a7f5e0d None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:26.686812 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b17e9-1795-43da-8699-5c854a7f5e0d None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:26.692732 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec146a6-3edd-46ea-8666-dcfde716ded3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:26.693457 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec146a6-3edd-46ea-8666-dcfde716ded3 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:26.693738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec146a6-3edd-46ea-8666-dcfde716ded3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:26.694029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec146a6-3edd-46ea-8666-dcfde716ded3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:26.694713 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec146a6-3edd-46ea-8666-dcfde716ded3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:26.696694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec146a6-3edd-46ea-8666-dcfde716ded3 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:26.738931 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1069/2119] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:35:26 2022] POST /identity/v3/projects => generated 406 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:35:26.743779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554af01f-7df2-4b00-8219-87a5e5b0fe39 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:26.755173 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554af01f-7df2-4b00-8219-87a5e5b0fe39 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:26.832158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b1cb6-9d6a-483e-a329-139ac7cbefe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:26.833116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b1cb6-9d6a-483e-a329-139ac7cbefe0 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:26.833116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b1cb6-9d6a-483e-a329-139ac7cbefe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:26.833347 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b1cb6-9d6a-483e-a329-139ac7cbefe0 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:26.834057 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b1cb6-9d6a-483e-a329-139ac7cbefe0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:26.836130 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b1cb6-9d6a-483e-a329-139ac7cbefe0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:26.844782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:26.868356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1051/2120] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:35:26 2022] POST /identity/v3/users => generated 304 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:35:26.873151 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec146a6-3edd-46ea-8666-dcfde716ded3 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:26.888973 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec146a6-3edd-46ea-8666-dcfde716ded3 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:26.897912 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa42458-a607-4744-b4cb-5587f4a7cfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:26.899136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa42458-a607-4744-b4cb-5587f4a7cfa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:26.899714 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa42458-a607-4744-b4cb-5587f4a7cfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:26.900453 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa42458-a607-4744-b4cb-5587f4a7cfa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:26.901244 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa42458-a607-4744-b4cb-5587f4a7cfa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:26.903083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa42458-a607-4744-b4cb-5587f4a7cfa6 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:26.916431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1070/2121] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:35:26 2022] GET /identity/v3/roles => generated 1427 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:35:26.920518 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b1cb6-9d6a-483e-a329-139ac7cbefe0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:26.933243 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b1cb6-9d6a-483e-a329-139ac7cbefe0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:26.939357 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fdfad-eb05-457c-ae13-e795da88f035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:26.939884 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fdfad-eb05-457c-ae13-e795da88f035 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:26.940173 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fdfad-eb05-457c-ae13-e795da88f035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:26.940444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fdfad-eb05-457c-ae13-e795da88f035 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:26.941180 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fdfad-eb05-457c-ae13-e795da88f035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:26.942973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fdfad-eb05-457c-ae13-e795da88f035 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:26.956417 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1052/2122] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:35:26 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:35:26.961835 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa42458-a607-4744-b4cb-5587f4a7cfa6 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:26.974947 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa42458-a607-4744-b4cb-5587f4a7cfa6 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:26.992214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da3d19-cd4c-45f8-9e8b-c7e20ea37596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:26.992214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da3d19-cd4c-45f8-9e8b-c7e20ea37596 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:26.992214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da3d19-cd4c-45f8-9e8b-c7e20ea37596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:26.992214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da3d19-cd4c-45f8-9e8b-c7e20ea37596 admin admin] PATH_INFO: `/v3/projects/240826d54dd54003941dea72cb65ece3/users/4e99f4328d4941a19ae14daaa43c1560/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:27.012581 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3da3d19-cd4c-45f8-9e8b-c7e20ea37596 admin admin] RBAC: Authorizing `identity:create_grant(project_id=240826d54dd54003941dea72cb65ece3, user_id=4e99f4328d4941a19ae14daaa43c1560, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:27.015526 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3da3d19-cd4c-45f8-9e8b-c7e20ea37596 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:27.038666 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1071/2123] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:35:26 2022] PUT /identity/v3/projects/240826d54dd54003941dea72cb65ece3/users/4e99f4328d4941a19ae14daaa43c1560/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:35:27.050625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b33a0-797a-4253-bd19-cf143fab8e77 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:27.050995 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b33a0-797a-4253-bd19-cf143fab8e77 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:27.051205 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b33a0-797a-4253-bd19-cf143fab8e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:27.107852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7f9b33a0-797a-4253-bd19-cf143fab8e77 None None] MFA Rules not processed for user `4e99f4328d4941a19ae14daaa43c1560`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:27.113969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7f9b33a0-797a-4253-bd19-cf143fab8e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:27.219657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1053/2124] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:35:27 2022] POST /identity/v3/auth/tokens => generated 3292 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:35:27.225019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da3d19-cd4c-45f8-9e8b-c7e20ea37596 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:27.320997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da3d19-cd4c-45f8-9e8b-c7e20ea37596 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:27.326433 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d84dad-9663-44de-a77f-ff7b5cb1f9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:27.327151 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d84dad-9663-44de-a77f-ff7b5cb1f9f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:27.327440 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d84dad-9663-44de-a77f-ff7b5cb1f9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:27.327715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d84dad-9663-44de-a77f-ff7b5cb1f9f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:27.328494 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d84dad-9663-44de-a77f-ff7b5cb1f9f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:27.330517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d84dad-9663-44de-a77f-ff7b5cb1f9f0 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:27.363879 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1072/2125] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:35:27 2022] POST /identity/v3/projects => generated 402 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:35:27.368284 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b33a0-797a-4253-bd19-cf143fab8e77 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:27.382669 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b33a0-797a-4253-bd19-cf143fab8e77 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:27.467862 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2aa0f-7c14-419b-ba6f-a5c3721a6bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:27.468678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2aa0f-7c14-419b-ba6f-a5c3721a6bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:27.469049 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2aa0f-7c14-419b-ba6f-a5c3721a6bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:27.469328 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2aa0f-7c14-419b-ba6f-a5c3721a6bb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:27.470029 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2aa0f-7c14-419b-ba6f-a5c3721a6bb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:27.472067 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2aa0f-7c14-419b-ba6f-a5c3721a6bb0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:27.498268 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1054/2126] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:35:27 2022] POST /identity/v3/users => generated 308 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:35:27.503253 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d84dad-9663-44de-a77f-ff7b5cb1f9f0 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:27.513254 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:27.521566 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d84dad-9663-44de-a77f-ff7b5cb1f9f0 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:27.532319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd12204-f027-4310-abd0-f9641365cc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:27.533221 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd12204-f027-4310-abd0-f9641365cc7a admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:27.533487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd12204-f027-4310-abd0-f9641365cc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:27.533969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd12204-f027-4310-abd0-f9641365cc7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:27.534323 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd12204-f027-4310-abd0-f9641365cc7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:27.537232 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd12204-f027-4310-abd0-f9641365cc7a admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:27.561498 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1073/2127] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:35:27 2022] GET /identity/v3/roles => generated 1427 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:35:27.568939 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2aa0f-7c14-419b-ba6f-a5c3721a6bb0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:27.587458 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2aa0f-7c14-419b-ba6f-a5c3721a6bb0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:27.596396 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2b3aa-1e7e-44b9-ad92-434eec9b6bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:27.597280 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2b3aa-1e7e-44b9-ad92-434eec9b6bc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:27.597693 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2b3aa-1e7e-44b9-ad92-434eec9b6bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:27.598048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2b3aa-1e7e-44b9-ad92-434eec9b6bc4 admin admin] PATH_INFO: `/v3/projects/bfac602a4a624c569036def52a849a08/users/eb293b800e0c4e03b5afae03b16e4aa2/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:27.627342 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa2b3aa-1e7e-44b9-ad92-434eec9b6bc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfac602a4a624c569036def52a849a08, user_id=eb293b800e0c4e03b5afae03b16e4aa2, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:27.630271 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa2b3aa-1e7e-44b9-ad92-434eec9b6bc4 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:27.647983 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1055/2128] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:35:27 2022] PUT /identity/v3/projects/bfac602a4a624c569036def52a849a08/users/eb293b800e0c4e03b5afae03b16e4aa2/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:35:27.653027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd12204-f027-4310-abd0-f9641365cc7a admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:27.664713 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd12204-f027-4310-abd0-f9641365cc7a admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:27.735684 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5bd747-c3a2-4fa9-b307-d03faa821650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:27.736383 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5bd747-c3a2-4fa9-b307-d03faa821650 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:27.736740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5bd747-c3a2-4fa9-b307-d03faa821650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:27.737052 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5bd747-c3a2-4fa9-b307-d03faa821650 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:27.737884 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5bd747-c3a2-4fa9-b307-d03faa821650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:27.739649 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5bd747-c3a2-4fa9-b307-d03faa821650 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:27.755364 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1074/2129] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:35:27 2022] GET /identity/v3/roles => generated 1427 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:35:27.761155 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2b3aa-1e7e-44b9-ad92-434eec9b6bc4 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:27.775577 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2b3aa-1e7e-44b9-ad92-434eec9b6bc4 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:27.781326 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe162f-a847-4190-80be-2f7b59f9d386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:27.782081 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe162f-a847-4190-80be-2f7b59f9d386 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:27.782365 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe162f-a847-4190-80be-2f7b59f9d386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:27.782654 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe162f-a847-4190-80be-2f7b59f9d386 admin admin] PATH_INFO: `/v3/projects/bfac602a4a624c569036def52a849a08/users/eb293b800e0c4e03b5afae03b16e4aa2/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:27.785307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe162f-a847-4190-80be-2f7b59f9d386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfac602a4a624c569036def52a849a08, user_id=eb293b800e0c4e03b5afae03b16e4aa2, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:27.788908 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe162f-a847-4190-80be-2f7b59f9d386 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:27.802313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1056/2130] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:35:27 2022] PUT /identity/v3/projects/bfac602a4a624c569036def52a849a08/users/eb293b800e0c4e03b5afae03b16e4aa2/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:35:27.815428 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ee48b-a83a-433b-8259-e9280bf63496 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:27.815731 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ee48b-a83a-433b-8259-e9280bf63496 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:27.816242 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ee48b-a83a-433b-8259-e9280bf63496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:27.851403 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-6c2ee48b-a83a-433b-8259-e9280bf63496 None None] MFA Rules not processed for user `eb293b800e0c4e03b5afae03b16e4aa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:27.853538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6c2ee48b-a83a-433b-8259-e9280bf63496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:27.948301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1075/2131] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:35:27 2022] POST /identity/v3/auth/tokens => generated 3355 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:35:27.961211 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe162f-a847-4190-80be-2f7b59f9d386 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:27.963828 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-88fe162f-a847-4190-80be-2f7b59f9d386 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:27.991388 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:28.060292 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe162f-a847-4190-80be-2f7b59f9d386 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:28.068228 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba2b4f-2262-436b-9cdb-da9c37769a93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:28.068914 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba2b4f-2262-436b-9cdb-da9c37769a93 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:28.069205 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba2b4f-2262-436b-9cdb-da9c37769a93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:28.069487 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba2b4f-2262-436b-9cdb-da9c37769a93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:28.080580 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba2b4f-2262-436b-9cdb-da9c37769a93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:28.082262 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba2b4f-2262-436b-9cdb-da9c37769a93 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:28.096607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1057/2132] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:35:27 2022] GET /identity/v3/auth/tokens => generated 3355 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:35:28.498595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ee48b-a83a-433b-8259-e9280bf63496 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:28.576093 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ee48b-a83a-433b-8259-e9280bf63496 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:28.584029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63c618-cfad-4cde-8667-29ddbb7b9067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:28.584029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63c618-cfad-4cde-8667-29ddbb7b9067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:28.584029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63c618-cfad-4cde-8667-29ddbb7b9067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:28.584029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63c618-cfad-4cde-8667-29ddbb7b9067 admin admin] PATH_INFO: `/v3/users/4e99f4328d4941a19ae14daaa43c1560` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:28.585128 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb63c618-cfad-4cde-8667-29ddbb7b9067 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e99f4328d4941a19ae14daaa43c1560)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:28.587119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb63c618-cfad-4cde-8667-29ddbb7b9067 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:28.605543 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:28.645488 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-fb63c618-cfad-4cde-8667-29ddbb7b9067 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e99f4328d4941a19ae14daaa43c1560'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:28.646246 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-fb63c618-cfad-4cde-8667-29ddbb7b9067 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e99f4328d4941a19ae14daaa43c1560'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:28.661617 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-fb63c618-cfad-4cde-8667-29ddbb7b9067 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e99f4328d4941a19ae14daaa43c1560'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:28.669362 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-fb63c618-cfad-4cde-8667-29ddbb7b9067 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e99f4328d4941a19ae14daaa43c1560'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:28.683927 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-fb63c618-cfad-4cde-8667-29ddbb7b9067 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e99f4328d4941a19ae14daaa43c1560'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:28.694788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1076/2133] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:35:28 2022] DELETE /identity/v3/users/4e99f4328d4941a19ae14daaa43c1560 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:35:28.700312 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f4f60-17ee-4864-8890-d79952803e60 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:28.700623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f4f60-17ee-4864-8890-d79952803e60 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:28.700883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f4f60-17ee-4864-8890-d79952803e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:28.724639 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-db4f4f60-17ee-4864-8890-d79952803e60 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:28.725903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-db4f4f60-17ee-4864-8890-d79952803e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:28.802247 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1058/2134] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:35:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:35:28.818340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63c618-cfad-4cde-8667-29ddbb7b9067 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:28.820512 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-fb63c618-cfad-4cde-8667-29ddbb7b9067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:28.886974 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63c618-cfad-4cde-8667-29ddbb7b9067 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:28.891343 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcad1c5-ab83-4cb8-ae2a-1d9b890ec30c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:28.892124 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcad1c5-ab83-4cb8-ae2a-1d9b890ec30c service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:28.892547 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcad1c5-ab83-4cb8-ae2a-1d9b890ec30c service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:28.892658 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcad1c5-ab83-4cb8-ae2a-1d9b890ec30c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:28.903747 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcad1c5-ab83-4cb8-ae2a-1d9b890ec30c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:28.905282 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcad1c5-ab83-4cb8-ae2a-1d9b890ec30c service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:28.916522 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1077/2135] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:35:28 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:35:29.253162 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f4f60-17ee-4864-8890-d79952803e60 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:29.256396 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-db4f4f60-17ee-4864-8890-d79952803e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:29.276499 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f4f60-17ee-4864-8890-d79952803e60 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:29.286874 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc8b21-4b04-49b2-9cdf-4c0f70c97abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:29.287624 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc8b21-4b04-49b2-9cdf-4c0f70c97abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:29.287900 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc8b21-4b04-49b2-9cdf-4c0f70c97abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:29.288214 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc8b21-4b04-49b2-9cdf-4c0f70c97abd admin admin] PATH_INFO: `/v3/projects/240826d54dd54003941dea72cb65ece3` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:29.289614 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc8b21-4b04-49b2-9cdf-4c0f70c97abd admin admin] RBAC: Authorizing `identity:delete_project(project_id=240826d54dd54003941dea72cb65ece3)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:29.291616 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc8b21-4b04-49b2-9cdf-4c0f70c97abd admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:29.366738 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cecc8b21-4b04-49b2-9cdf-4c0f70c97abd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '240826d54dd54003941dea72cb65ece3', 'request_id': 'req-cecc8b21-4b04-49b2-9cdf-4c0f70c97abd'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:29.373616 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cecc8b21-4b04-49b2-9cdf-4c0f70c97abd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '240826d54dd54003941dea72cb65ece3', 'request_id': 'req-cecc8b21-4b04-49b2-9cdf-4c0f70c97abd'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:29.380037 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cecc8b21-4b04-49b2-9cdf-4c0f70c97abd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '240826d54dd54003941dea72cb65ece3', 'request_id': 'req-cecc8b21-4b04-49b2-9cdf-4c0f70c97abd'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:29.396654 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cecc8b21-4b04-49b2-9cdf-4c0f70c97abd admin admin] The token cache is being invalidate because project 240826d54dd54003941dea72cb65ece3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:35:29.397205 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cecc8b21-4b04-49b2-9cdf-4c0f70c97abd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:29.398055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cecc8b21-4b04-49b2-9cdf-4c0f70c97abd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:29.399871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1059/2136] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:35:29 2022] DELETE /identity/v3/projects/240826d54dd54003941dea72cb65ece3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:35:29.404430 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcad1c5-ab83-4cb8-ae2a-1d9b890ec30c service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:29.416075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcad1c5-ab83-4cb8-ae2a-1d9b890ec30c service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:29.495839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb2b5b-71b4-4d45-a28d-b4b8aecbff6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:29.496576 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb2b5b-71b4-4d45-a28d-b4b8aecbff6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:29.496864 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb2b5b-71b4-4d45-a28d-b4b8aecbff6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:29.497150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb2b5b-71b4-4d45-a28d-b4b8aecbff6b admin admin] PATH_INFO: `/v3/users/eb293b800e0c4e03b5afae03b16e4aa2` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:29.498449 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb2b5b-71b4-4d45-a28d-b4b8aecbff6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb293b800e0c4e03b5afae03b16e4aa2)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:29.500893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb2b5b-71b4-4d45-a28d-b4b8aecbff6b admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:29.569143 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:29.578684 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-2dbb2b5b-71b4-4d45-a28d-b4b8aecbff6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb293b800e0c4e03b5afae03b16e4aa2'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:29.579556 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-2dbb2b5b-71b4-4d45-a28d-b4b8aecbff6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb293b800e0c4e03b5afae03b16e4aa2'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:29.600197 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-2dbb2b5b-71b4-4d45-a28d-b4b8aecbff6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb293b800e0c4e03b5afae03b16e4aa2'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:29.625902 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-2dbb2b5b-71b4-4d45-a28d-b4b8aecbff6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb293b800e0c4e03b5afae03b16e4aa2'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:29.636437 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-2dbb2b5b-71b4-4d45-a28d-b4b8aecbff6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb293b800e0c4e03b5afae03b16e4aa2'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:29.645009 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1078/2137] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:35:29 2022] DELETE /identity/v3/users/eb293b800e0c4e03b5afae03b16e4aa2 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:35:30.079320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc8b21-4b04-49b2-9cdf-4c0f70c97abd admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:30.082635 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cecc8b21-4b04-49b2-9cdf-4c0f70c97abd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:30.107731 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc8b21-4b04-49b2-9cdf-4c0f70c97abd admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:30.173032 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:30.195171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e1bf6-7ff4-4258-90b7-cf0d957164b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:30.195800 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e1bf6-7ff4-4258-90b7-cf0d957164b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:30.196141 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e1bf6-7ff4-4258-90b7-cf0d957164b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:30.196458 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e1bf6-7ff4-4258-90b7-cf0d957164b9 admin admin] PATH_INFO: `/v3/projects/bfac602a4a624c569036def52a849a08` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:30.197880 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e1bf6-7ff4-4258-90b7-cf0d957164b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfac602a4a624c569036def52a849a08)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:30.199864 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e1bf6-7ff4-4258-90b7-cf0d957164b9 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:30.262505 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b51e1bf6-7ff4-4258-90b7-cf0d957164b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfac602a4a624c569036def52a849a08', 'request_id': 'req-b51e1bf6-7ff4-4258-90b7-cf0d957164b9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:30.270145 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b51e1bf6-7ff4-4258-90b7-cf0d957164b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfac602a4a624c569036def52a849a08', 'request_id': 'req-b51e1bf6-7ff4-4258-90b7-cf0d957164b9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:30.277240 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b51e1bf6-7ff4-4258-90b7-cf0d957164b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfac602a4a624c569036def52a849a08', 'request_id': 'req-b51e1bf6-7ff4-4258-90b7-cf0d957164b9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:30.295333 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b51e1bf6-7ff4-4258-90b7-cf0d957164b9 admin admin] The token cache is being invalidate because project bfac602a4a624c569036def52a849a08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:35:30.295727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b51e1bf6-7ff4-4258-90b7-cf0d957164b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:30.296375 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b51e1bf6-7ff4-4258-90b7-cf0d957164b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:35:30.298255 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1060/2138] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:35:30 2022] DELETE /identity/v3/projects/bfac602a4a624c569036def52a849a08 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:35:30.305413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3e86a-f0f5-4e24-a6cf-6a15601b2732 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:30.305413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3e86a-f0f5-4e24-a6cf-6a15601b2732 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:30.306942 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3e86a-f0f5-4e24-a6cf-6a15601b2732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:30.329245 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-02a3e86a-f0f5-4e24-a6cf-6a15601b2732 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:30.331372 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-02a3e86a-f0f5-4e24-a6cf-6a15601b2732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:30.403715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1079/2139] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:35:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:35:30.413248 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18f69c-9b78-415a-9a93-f1db8fd37ebb None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:30.413665 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18f69c-9b78-415a-9a93-f1db8fd37ebb None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:30.414054 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18f69c-9b78-415a-9a93-f1db8fd37ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:30.450667 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ab18f69c-9b78-415a-9a93-f1db8fd37ebb None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:30.451865 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ab18f69c-9b78-415a-9a93-f1db8fd37ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:30.460803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1061/2140] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:35:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:35:30.465783 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3e86a-f0f5-4e24-a6cf-6a15601b2732 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:30.477700 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3e86a-f0f5-4e24-a6cf-6a15601b2732 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:30.483047 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0845259-e64b-4bef-b6dd-ec2e38bf70a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:30.483703 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0845259-e64b-4bef-b6dd-ec2e38bf70a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:30.484020 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0845259-e64b-4bef-b6dd-ec2e38bf70a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:30.484297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0845259-e64b-4bef-b6dd-ec2e38bf70a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:30.485025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0845259-e64b-4bef-b6dd-ec2e38bf70a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:35:30.485324 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0845259-e64b-4bef-b6dd-ec2e38bf70a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:30.487010 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0845259-e64b-4bef-b6dd-ec2e38bf70a8 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:30.502794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1080/2141] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:35:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:35:30.507358 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18f69c-9b78-415a-9a93-f1db8fd37ebb None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:30.518298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18f69c-9b78-415a-9a93-f1db8fd37ebb None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:30.523670 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0a8d4-3db2-4ca8-af34-8f6a2d572dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:30.524361 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0a8d4-3db2-4ca8-af34-8f6a2d572dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:30.524645 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0a8d4-3db2-4ca8-af34-8f6a2d572dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:30.524938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0a8d4-3db2-4ca8-af34-8f6a2d572dd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:30.525632 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb0a8d4-3db2-4ca8-af34-8f6a2d572dd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:30.527615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb0a8d4-3db2-4ca8-af34-8f6a2d572dd7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:30.571223 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1062/2142] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:35:30 2022] POST /identity/v3/projects => generated 388 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:35:30.590914 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0845259-e64b-4bef-b6dd-ec2e38bf70a8 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:30.604544 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0845259-e64b-4bef-b6dd-ec2e38bf70a8 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:30.669209 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d5718-877e-4766-866f-8581cba7ed44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:30.669809 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d5718-877e-4766-866f-8581cba7ed44 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:30.670131 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d5718-877e-4766-866f-8581cba7ed44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:30.670553 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d5718-877e-4766-866f-8581cba7ed44 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:30.671249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d5718-877e-4766-866f-8581cba7ed44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:30.673375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d5718-877e-4766-866f-8581cba7ed44 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:30.699237 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1081/2143] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:35:30 2022] POST /identity/v3/users => generated 295 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:35:30.705345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0a8d4-3db2-4ca8-af34-8f6a2d572dd7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:30.717533 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0a8d4-3db2-4ca8-af34-8f6a2d572dd7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:30.723728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b34571-9d52-48a7-8710-963f9525f4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:30.724486 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b34571-9d52-48a7-8710-963f9525f4fe admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:30.724909 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b34571-9d52-48a7-8710-963f9525f4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:30.725417 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b34571-9d52-48a7-8710-963f9525f4fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:30.726643 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b34571-9d52-48a7-8710-963f9525f4fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:30.728284 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b34571-9d52-48a7-8710-963f9525f4fe admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:30.740977 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1063/2144] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:35:30 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:35:30.746326 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d5718-877e-4766-866f-8581cba7ed44 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:30.760779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d5718-877e-4766-866f-8581cba7ed44 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:30.766145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9a718-8db6-4b0f-8015-4e349cd27a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:30.768775 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9a718-8db6-4b0f-8015-4e349cd27a23 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:30.769073 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9a718-8db6-4b0f-8015-4e349cd27a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:30.769360 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9a718-8db6-4b0f-8015-4e349cd27a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:30.770100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9a718-8db6-4b0f-8015-4e349cd27a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:30.771781 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9a718-8db6-4b0f-8015-4e349cd27a23 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:30.784621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1082/2145] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:35:30 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:35:30.790767 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b34571-9d52-48a7-8710-963f9525f4fe admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:30.804627 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b34571-9d52-48a7-8710-963f9525f4fe admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:30.810382 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d3c83-6f13-490b-ba6d-8097a8269b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:30.811130 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d3c83-6f13-490b-ba6d-8097a8269b4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:30.812203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d3c83-6f13-490b-ba6d-8097a8269b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:30.812203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d3c83-6f13-490b-ba6d-8097a8269b4a admin admin] PATH_INFO: `/v3/projects/862c4b20059e4058bdad868151fc2539/users/a5bae60f33b04a48a1b3ea5179f481b6/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:30.831089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d3c83-6f13-490b-ba6d-8097a8269b4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=862c4b20059e4058bdad868151fc2539, user_id=a5bae60f33b04a48a1b3ea5179f481b6, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:30.834440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d3c83-6f13-490b-ba6d-8097a8269b4a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:30.861708 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1064/2146] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:35:30 2022] PUT /identity/v3/projects/862c4b20059e4058bdad868151fc2539/users/a5bae60f33b04a48a1b3ea5179f481b6/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:35:30.874267 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73657c68-5cb5-40ee-b426-9d40a52fcf9e None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:30.874636 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73657c68-5cb5-40ee-b426-9d40a52fcf9e None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:30.874842 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73657c68-5cb5-40ee-b426-9d40a52fcf9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:30.909299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-73657c68-5cb5-40ee-b426-9d40a52fcf9e None None] MFA Rules not processed for user `a5bae60f33b04a48a1b3ea5179f481b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:30.910690 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-73657c68-5cb5-40ee-b426-9d40a52fcf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:30.922406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:30.998656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1083/2147] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:35:30 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:35:31.004217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d3c83-6f13-490b-ba6d-8097a8269b4a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:31.096993 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d3c83-6f13-490b-ba6d-8097a8269b4a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:31.107636 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414019e6-7b4f-47bd-b748-61c25a2e7f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:31.107636 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414019e6-7b4f-47bd-b748-61c25a2e7f44 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:31.107636 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414019e6-7b4f-47bd-b748-61c25a2e7f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:31.107636 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414019e6-7b4f-47bd-b748-61c25a2e7f44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:31.108290 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414019e6-7b4f-47bd-b748-61c25a2e7f44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:31.108868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414019e6-7b4f-47bd-b748-61c25a2e7f44 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:31.134029 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:31.144704 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1065/2148] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:35:31 2022] POST /identity/v3/projects => generated 386 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:35:31.148818 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73657c68-5cb5-40ee-b426-9d40a52fcf9e None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:31.162160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73657c68-5cb5-40ee-b426-9d40a52fcf9e None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:31.226642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3629035-23e8-4f60-86f6-3313f54e5001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:31.227570 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3629035-23e8-4f60-86f6-3313f54e5001 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:31.227847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3629035-23e8-4f60-86f6-3313f54e5001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:31.228157 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3629035-23e8-4f60-86f6-3313f54e5001 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:31.228857 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3629035-23e8-4f60-86f6-3313f54e5001 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:31.230869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3629035-23e8-4f60-86f6-3313f54e5001 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:31.260701 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1084/2149] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:35:31 2022] POST /identity/v3/users => generated 294 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:35:31.265208 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414019e6-7b4f-47bd-b748-61c25a2e7f44 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:31.283349 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414019e6-7b4f-47bd-b748-61c25a2e7f44 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:31.308130 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4973e-8786-485b-83a0-2aeefbf2ca03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:31.309028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4973e-8786-485b-83a0-2aeefbf2ca03 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:31.309320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4973e-8786-485b-83a0-2aeefbf2ca03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:31.309601 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4973e-8786-485b-83a0-2aeefbf2ca03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:31.310496 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f4973e-8786-485b-83a0-2aeefbf2ca03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:31.319885 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f4973e-8786-485b-83a0-2aeefbf2ca03 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:31.337950 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1066/2150] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:35:31 2022] GET /identity/v3/roles => generated 1427 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:35:31.343038 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3629035-23e8-4f60-86f6-3313f54e5001 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:31.363057 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3629035-23e8-4f60-86f6-3313f54e5001 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:31.372994 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5049c3-b691-427b-8ca9-5d05804ca073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:31.374050 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5049c3-b691-427b-8ca9-5d05804ca073 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:31.374516 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5049c3-b691-427b-8ca9-5d05804ca073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:31.374988 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5049c3-b691-427b-8ca9-5d05804ca073 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:31.376310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5049c3-b691-427b-8ca9-5d05804ca073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:31.380126 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5049c3-b691-427b-8ca9-5d05804ca073 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:31.405108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1085/2151] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:35:31 2022] GET /identity/v3/roles => generated 1427 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:35:31.406927 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4973e-8786-485b-83a0-2aeefbf2ca03 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:31.424687 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4973e-8786-485b-83a0-2aeefbf2ca03 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:31.436611 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e549b8-ac27-46d7-b1db-5e5a139fd402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:31.437381 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e549b8-ac27-46d7-b1db-5e5a139fd402 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:31.437762 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e549b8-ac27-46d7-b1db-5e5a139fd402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:31.438196 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e549b8-ac27-46d7-b1db-5e5a139fd402 admin admin] PATH_INFO: `/v3/projects/bc84aba9a68b4ba08e238101073c6374/users/8ab110555913420a94a5a3508f48de62/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:31.464568 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e549b8-ac27-46d7-b1db-5e5a139fd402 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc84aba9a68b4ba08e238101073c6374, user_id=8ab110555913420a94a5a3508f48de62, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:31.467190 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e549b8-ac27-46d7-b1db-5e5a139fd402 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:31.485710 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1067/2152] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:35:31 2022] PUT /identity/v3/projects/bc84aba9a68b4ba08e238101073c6374/users/8ab110555913420a94a5a3508f48de62/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:35:31.497284 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c3c6b-6db3-4aec-ba31-b4cee3e0e921 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:31.497787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c3c6b-6db3-4aec-ba31-b4cee3e0e921 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:31.498076 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c3c6b-6db3-4aec-ba31-b4cee3e0e921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:31.541464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-fb2c3c6b-6db3-4aec-ba31-b4cee3e0e921 None None] MFA Rules not processed for user `8ab110555913420a94a5a3508f48de62`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:31.545248 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-fb2c3c6b-6db3-4aec-ba31-b4cee3e0e921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:31.664854 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1086/2153] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:35:31 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:35:31.669607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e549b8-ac27-46d7-b1db-5e5a139fd402 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:31.767900 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e549b8-ac27-46d7-b1db-5e5a139fd402 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:31.774069 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d32a2b-278c-4304-bfe0-e896a661c680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:31.774893 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d32a2b-278c-4304-bfe0-e896a661c680 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:31.775299 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d32a2b-278c-4304-bfe0-e896a661c680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:31.775684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d32a2b-278c-4304-bfe0-e896a661c680 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:31.776608 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d32a2b-278c-4304-bfe0-e896a661c680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:31.778693 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d32a2b-278c-4304-bfe0-e896a661c680 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:31.826816 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1068/2154] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:35:31 2022] POST /identity/v3/projects => generated 386 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:35:31.831476 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c3c6b-6db3-4aec-ba31-b4cee3e0e921 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:31.843728 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c3c6b-6db3-4aec-ba31-b4cee3e0e921 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:31.864554 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:31.948748 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6be901-c2da-4da9-b326-a7cc087e5bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:31.950101 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6be901-c2da-4da9-b326-a7cc087e5bbe admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:31.950596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6be901-c2da-4da9-b326-a7cc087e5bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:31.951131 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6be901-c2da-4da9-b326-a7cc087e5bbe admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:31.952494 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6be901-c2da-4da9-b326-a7cc087e5bbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:31.956670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6be901-c2da-4da9-b326-a7cc087e5bbe admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:31.998925 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1087/2155] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:35:31 2022] POST /identity/v3/users => generated 300 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:35:32.006210 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d32a2b-278c-4304-bfe0-e896a661c680 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:32.019408 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d32a2b-278c-4304-bfe0-e896a661c680 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:32.025262 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507561b2-6903-4277-a2b9-ab35c2e02378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:32.027095 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507561b2-6903-4277-a2b9-ab35c2e02378 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:32.027427 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507561b2-6903-4277-a2b9-ab35c2e02378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:32.027763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507561b2-6903-4277-a2b9-ab35c2e02378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:32.028965 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507561b2-6903-4277-a2b9-ab35c2e02378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:32.031412 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507561b2-6903-4277-a2b9-ab35c2e02378 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:32.051540 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1069/2156] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:35:32 2022] GET /identity/v3/roles => generated 1427 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:35:32.056490 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6be901-c2da-4da9-b326-a7cc087e5bbe admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:32.071669 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6be901-c2da-4da9-b326-a7cc087e5bbe admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:32.080674 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9824862-060d-44d9-ac88-ab5b35ce957f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:32.081791 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9824862-060d-44d9-ac88-ab5b35ce957f admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:32.082267 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9824862-060d-44d9-ac88-ab5b35ce957f admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:32.082765 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9824862-060d-44d9-ac88-ab5b35ce957f admin admin] PATH_INFO: `/v3/projects/4770e648d6d649798edbaf66702d08f9/users/d62622b727e44c39a23a75478e9ca177/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:32.120096 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9824862-060d-44d9-ac88-ab5b35ce957f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4770e648d6d649798edbaf66702d08f9, user_id=d62622b727e44c39a23a75478e9ca177, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:32.125865 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9824862-060d-44d9-ac88-ab5b35ce957f admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:32.148813 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1088/2157] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:35:32 2022] PUT /identity/v3/projects/4770e648d6d649798edbaf66702d08f9/users/d62622b727e44c39a23a75478e9ca177/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:35:32.153880 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507561b2-6903-4277-a2b9-ab35c2e02378 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:32.167326 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507561b2-6903-4277-a2b9-ab35c2e02378 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:32.281477 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d554bf-9c3a-42ea-b060-20f1f53153e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:32.282237 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d554bf-9c3a-42ea-b060-20f1f53153e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:32.282462 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d554bf-9c3a-42ea-b060-20f1f53153e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:32.283050 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d554bf-9c3a-42ea-b060-20f1f53153e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:32.284248 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d554bf-9c3a-42ea-b060-20f1f53153e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:32.287497 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d554bf-9c3a-42ea-b060-20f1f53153e1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:32.311921 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9824862-060d-44d9-ac88-ab5b35ce957f admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:32.313343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1070/2158] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:35:32 2022] GET /identity/v3/roles => generated 1427 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:35:32.330630 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9824862-060d-44d9-ac88-ab5b35ce957f admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:32.338456 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41d3ce-5dae-4723-b3d3-0c0323375bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:32.339650 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41d3ce-5dae-4723-b3d3-0c0323375bed admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:32.340187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41d3ce-5dae-4723-b3d3-0c0323375bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:32.340667 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41d3ce-5dae-4723-b3d3-0c0323375bed admin admin] PATH_INFO: `/v3/projects/4770e648d6d649798edbaf66702d08f9/users/d62622b727e44c39a23a75478e9ca177/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:32.344419 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb41d3ce-5dae-4723-b3d3-0c0323375bed admin admin] RBAC: Authorizing `identity:create_grant(project_id=4770e648d6d649798edbaf66702d08f9, user_id=d62622b727e44c39a23a75478e9ca177, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:32.347342 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb41d3ce-5dae-4723-b3d3-0c0323375bed admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:32.362898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1089/2159] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:35:32 2022] PUT /identity/v3/projects/4770e648d6d649798edbaf66702d08f9/users/d62622b727e44c39a23a75478e9ca177/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:35:32.373866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33397ba6-c789-4865-9c4e-44590a6a3405 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:32.374140 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33397ba6-c789-4865-9c4e-44590a6a3405 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:32.374403 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33397ba6-c789-4865-9c4e-44590a6a3405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:32.424957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-33397ba6-c789-4865-9c4e-44590a6a3405 None None] MFA Rules not processed for user `d62622b727e44c39a23a75478e9ca177`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:32.426335 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-33397ba6-c789-4865-9c4e-44590a6a3405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:32.455065 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:32.513693 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1071/2160] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:35:32 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:35:32.525717 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41d3ce-5dae-4723-b3d3-0c0323375bed admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:32.528277 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-fb41d3ce-5dae-4723-b3d3-0c0323375bed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:32.594899 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41d3ce-5dae-4723-b3d3-0c0323375bed admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:32.599670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2e095-5426-4ab4-895c-9988b067cc5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:32.600451 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2e095-5426-4ab4-895c-9988b067cc5c service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:32.600626 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2e095-5426-4ab4-895c-9988b067cc5c service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:32.600898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2e095-5426-4ab4-895c-9988b067cc5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:32.613623 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2e095-5426-4ab4-895c-9988b067cc5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:32.615494 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2e095-5426-4ab4-895c-9988b067cc5c service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:32.627882 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1090/2161] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:35:32 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:35:33.328842 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33397ba6-c789-4865-9c4e-44590a6a3405 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:33.340335 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33397ba6-c789-4865-9c4e-44590a6a3405 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:33.357914 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02964e15-f2a2-4850-9c52-f158ba1c07e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:33.359157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02964e15-f2a2-4850-9c52-f158ba1c07e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:33.359836 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02964e15-f2a2-4850-9c52-f158ba1c07e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:33.360344 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02964e15-f2a2-4850-9c52-f158ba1c07e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:33.453995 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02964e15-f2a2-4850-9c52-f158ba1c07e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:33.456135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02964e15-f2a2-4850-9c52-f158ba1c07e5 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:33.469763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1072/2162] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:35:33 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:35:42.621881 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=61,keystone:INSERT=5 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:43.463310 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=47 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:35:53.281823 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2e095-5426-4ab4-895c-9988b067cc5c service cinder] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:53.300716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2e095-5426-4ab4-895c-9988b067cc5c service cinder] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:53.307722 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10921d-a7ef-4ad0-be05-1b488c2bb4d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:53.308709 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10921d-a7ef-4ad0-be05-1b488c2bb4d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:53.309125 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10921d-a7ef-4ad0-be05-1b488c2bb4d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:53.309569 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10921d-a7ef-4ad0-be05-1b488c2bb4d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:53.434160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10921d-a7ef-4ad0-be05-1b488c2bb4d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:53.435876 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10921d-a7ef-4ad0-be05-1b488c2bb4d6 service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:53.457253 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1091/2163] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:35:53 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:35:57.971235 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02964e15-f2a2-4850-9c52-f158ba1c07e5 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:57.975049 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-02964e15-f2a2-4850-9c52-f158ba1c07e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:58.168254 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02964e15-f2a2-4850-9c52-f158ba1c07e5 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:35:58.175521 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d629be96-af9e-4cdb-ae5e-348ea9c91178 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c7cfbe911c4378857848966f4f8d98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:35:58.178575 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d629be96-af9e-4cdb-ae5e-348ea9c91178 service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:58.178575 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d629be96-af9e-4cdb-ae5e-348ea9c91178 service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:58.178575 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d629be96-af9e-4cdb-ae5e-348ea9c91178 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:58.179970 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d629be96-af9e-4cdb-ae5e-348ea9c91178 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:58.193787 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d629be96-af9e-4cdb-ae5e-348ea9c91178 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:35:58.196163 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d629be96-af9e-4cdb-ae5e-348ea9c91178 service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:35:58.212351 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1073/2164] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:35:57 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:35:59.301270 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16355470-805d-47fe-902c-fba4fecc39a7 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:35:59.302127 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16355470-805d-47fe-902c-fba4fecc39a7 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:35:59.302127 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16355470-805d-47fe-902c-fba4fecc39a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:35:59.323727 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-16355470-805d-47fe-902c-fba4fecc39a7 None None] MFA Rules not processed for user `a025ea88db864b55b7537311d20f4808`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:35:59.325143 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-16355470-805d-47fe-902c-fba4fecc39a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:35:59.416435 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1092/2165] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:35:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:35:59.879916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d629be96-af9e-4cdb-ae5e-348ea9c91178 service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:35:59.882951 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d629be96-af9e-4cdb-ae5e-348ea9c91178 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:00.033419 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d629be96-af9e-4cdb-ae5e-348ea9c91178 service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:00.039586 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89715007-605f-407f-b0ef-4a949a43f23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:00.040571 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89715007-605f-407f-b0ef-4a949a43f23f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:00.041043 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89715007-605f-407f-b0ef-4a949a43f23f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:00.041458 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89715007-605f-407f-b0ef-4a949a43f23f admin admin] PATH_INFO: `/v3/users/078c26677e0340899511ac3af69544ff` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:00.043091 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89715007-605f-407f-b0ef-4a949a43f23f admin admin] RBAC: Authorizing `identity:delete_user(user_id=078c26677e0340899511ac3af69544ff)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:00.046037 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89715007-605f-407f-b0ef-4a949a43f23f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:00.166616 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-89715007-605f-407f-b0ef-4a949a43f23f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '078c26677e0340899511ac3af69544ff'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:00.180841 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-89715007-605f-407f-b0ef-4a949a43f23f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '078c26677e0340899511ac3af69544ff'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:00.181946 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-89715007-605f-407f-b0ef-4a949a43f23f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '078c26677e0340899511ac3af69544ff'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:00.213004 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-89715007-605f-407f-b0ef-4a949a43f23f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '078c26677e0340899511ac3af69544ff'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:00.214472 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-89715007-605f-407f-b0ef-4a949a43f23f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '078c26677e0340899511ac3af69544ff'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:00.247425 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1074/2166] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:35:59 2022] DELETE /identity/v3/users/078c26677e0340899511ac3af69544ff => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:00.255678 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd107e4a-6ab7-4f0a-a165-5723c7125306 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:00.256885 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd107e4a-6ab7-4f0a-a165-5723c7125306 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:00.257315 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd107e4a-6ab7-4f0a-a165-5723c7125306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:00.294927 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-bd107e4a-6ab7-4f0a-a165-5723c7125306 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:00.297966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-bd107e4a-6ab7-4f0a-a165-5723c7125306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:00.387223 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1093/2167] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:00 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:00.402896 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89715007-605f-407f-b0ef-4a949a43f23f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:00.405001 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-89715007-605f-407f-b0ef-4a949a43f23f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:00.422795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:00.495595 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89715007-605f-407f-b0ef-4a949a43f23f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:00.519116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc5f5c-0e24-4238-82e5-fca78842919c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:00.519823 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc5f5c-0e24-4238-82e5-fca78842919c service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:00.520168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc5f5c-0e24-4238-82e5-fca78842919c service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:00.524393 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc5f5c-0e24-4238-82e5-fca78842919c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:00.559623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc5f5c-0e24-4238-82e5-fca78842919c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:00.564627 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc5f5c-0e24-4238-82e5-fca78842919c service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:00.606211 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1075/2168] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:36:00 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:36:01.022722 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd107e4a-6ab7-4f0a-a165-5723c7125306 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:01.024664 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-bd107e4a-6ab7-4f0a-a165-5723c7125306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:01.044584 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd107e4a-6ab7-4f0a-a165-5723c7125306 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:01.054370 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a602c5-3ea6-4b23-ab0c-95ef3cdf9d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:01.056303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a602c5-3ea6-4b23-ab0c-95ef3cdf9d63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:01.057188 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a602c5-3ea6-4b23-ab0c-95ef3cdf9d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:01.057934 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a602c5-3ea6-4b23-ab0c-95ef3cdf9d63 admin admin] PATH_INFO: `/v3/users/a5bae60f33b04a48a1b3ea5179f481b6` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:01.061547 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a602c5-3ea6-4b23-ab0c-95ef3cdf9d63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5bae60f33b04a48a1b3ea5179f481b6)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:01.066034 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a602c5-3ea6-4b23-ab0c-95ef3cdf9d63 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:01.158826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:01.193289 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e5a602c5-3ea6-4b23-ab0c-95ef3cdf9d63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5bae60f33b04a48a1b3ea5179f481b6'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:01.194795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e5a602c5-3ea6-4b23-ab0c-95ef3cdf9d63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5bae60f33b04a48a1b3ea5179f481b6'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:01.221266 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e5a602c5-3ea6-4b23-ab0c-95ef3cdf9d63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5bae60f33b04a48a1b3ea5179f481b6'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:01.237930 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e5a602c5-3ea6-4b23-ab0c-95ef3cdf9d63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5bae60f33b04a48a1b3ea5179f481b6'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:01.252777 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e5a602c5-3ea6-4b23-ab0c-95ef3cdf9d63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5bae60f33b04a48a1b3ea5179f481b6'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:01.260347 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1094/2169] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:36:01 2022] DELETE /identity/v3/users/a5bae60f33b04a48a1b3ea5179f481b6 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:01.265703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f1b75-6a14-4c7a-9085-f34e9a576cb1 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:01.266313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f1b75-6a14-4c7a-9085-f34e9a576cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:01.266557 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f1b75-6a14-4c7a-9085-f34e9a576cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:01.323564 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-1d1f1b75-6a14-4c7a-9085-f34e9a576cb1 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:01.326128 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1d1f1b75-6a14-4c7a-9085-f34e9a576cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:01.332901 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a602c5-3ea6-4b23-ab0c-95ef3cdf9d63 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:01.335142 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-e5a602c5-3ea6-4b23-ab0c-95ef3cdf9d63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:01.462227 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1076/2170] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:01.467181 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a602c5-3ea6-4b23-ab0c-95ef3cdf9d63 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:01.473215 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a35e3c-f5c8-47ac-8b8b-c5e1f8f5eb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:01.474009 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a35e3c-f5c8-47ac-8b8b-c5e1f8f5eb65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:01.474370 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a35e3c-f5c8-47ac-8b8b-c5e1f8f5eb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:01.474959 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a35e3c-f5c8-47ac-8b8b-c5e1f8f5eb65 admin admin] PATH_INFO: `/v3/projects/7cf9298f363f46aa860b2bceef00d8a7` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:01.475826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f1b75-6a14-4c7a-9085-f34e9a576cb1 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:01.476466 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a35e3c-f5c8-47ac-8b8b-c5e1f8f5eb65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cf9298f363f46aa860b2bceef00d8a7)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:01.479047 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1d1f1b75-6a14-4c7a-9085-f34e9a576cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:01.479933 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a35e3c-f5c8-47ac-8b8b-c5e1f8f5eb65 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:01.555290 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f4a35e3c-f5c8-47ac-8b8b-c5e1f8f5eb65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cf9298f363f46aa860b2bceef00d8a7', 'request_id': 'req-f4a35e3c-f5c8-47ac-8b8b-c5e1f8f5eb65'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:01.567394 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f4a35e3c-f5c8-47ac-8b8b-c5e1f8f5eb65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cf9298f363f46aa860b2bceef00d8a7', 'request_id': 'req-f4a35e3c-f5c8-47ac-8b8b-c5e1f8f5eb65'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:01.574813 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f4a35e3c-f5c8-47ac-8b8b-c5e1f8f5eb65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cf9298f363f46aa860b2bceef00d8a7', 'request_id': 'req-f4a35e3c-f5c8-47ac-8b8b-c5e1f8f5eb65'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:01.578614 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f1b75-6a14-4c7a-9085-f34e9a576cb1 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:01.592048 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f4a35e3c-f5c8-47ac-8b8b-c5e1f8f5eb65 admin admin] The token cache is being invalidate because project 7cf9298f363f46aa860b2bceef00d8a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:36:01.592048 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f4a35e3c-f5c8-47ac-8b8b-c5e1f8f5eb65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:01.592692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f4a35e3c-f5c8-47ac-8b8b-c5e1f8f5eb65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:01.594785 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1095/2171] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:36:01 2022] DELETE /identity/v3/projects/7cf9298f363f46aa860b2bceef00d8a7 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:36:01.600136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deaa692-cfa1-44c0-8758-92e0378c0fe2 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:01.600433 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deaa692-cfa1-44c0-8758-92e0378c0fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:01.600693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deaa692-cfa1-44c0-8758-92e0378c0fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:01.624407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-6deaa692-cfa1-44c0-8758-92e0378c0fe2 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:01.625902 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6deaa692-cfa1-44c0-8758-92e0378c0fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:01.670049 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff7bc2-f635-4d78-85fe-42c55dcc0083 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:01.671213 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff7bc2-f635-4d78-85fe-42c55dcc0083 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:01.671657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff7bc2-f635-4d78-85fe-42c55dcc0083 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:01.672133 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff7bc2-f635-4d78-85fe-42c55dcc0083 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:01.675288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fdff7bc2-f635-4d78-85fe-42c55dcc0083 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:01.703660 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:01.715330 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:01.745145 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1096/2172] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:36:01.757472 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef61b9-a369-464a-b9ba-9c432be94434 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:01.758024 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef61b9-a369-464a-b9ba-9c432be94434 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:01.759104 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef61b9-a369-464a-b9ba-9c432be94434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:01.791507 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-83ef61b9-a369-464a-b9ba-9c432be94434 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:01.794453 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-83ef61b9-a369-464a-b9ba-9c432be94434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:01.814499 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1097/2173] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:01.816780 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef61b9-a369-464a-b9ba-9c432be94434 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:01.829527 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef61b9-a369-464a-b9ba-9c432be94434 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:01.833480 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff7bc2-f635-4d78-85fe-42c55dcc0083 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:01.836770 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff7bc2-f635-4d78-85fe-42c55dcc0083 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:01.841927 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f054e-2c71-4cde-a39e-c3b5faff9794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:01.843221 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f054e-2c71-4cde-a39e-c3b5faff9794 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:01.843592 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f054e-2c71-4cde-a39e-c3b5faff9794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:01.843878 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f054e-2c71-4cde-a39e-c3b5faff9794 admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:01.845426 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f054e-2c71-4cde-a39e-c3b5faff9794 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:36:01.845740 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f054e-2c71-4cde-a39e-c3b5faff9794 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:01.847693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f054e-2c71-4cde-a39e-c3b5faff9794 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:01.865643 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1077/2174] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:36:01 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:36:01.880799 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff7bc2-f635-4d78-85fe-42c55dcc0083 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:01.883638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1098/2175] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:36:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:36:01.900927 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff7bc2-f635-4d78-85fe-42c55dcc0083 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:01.914080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449f445-1d34-4ddd-81c5-9c771e44657b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:01.915305 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449f445-1d34-4ddd-81c5-9c771e44657b admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:01.915826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449f445-1d34-4ddd-81c5-9c771e44657b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:01.917104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449f445-1d34-4ddd-81c5-9c771e44657b admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:01.918482 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449f445-1d34-4ddd-81c5-9c771e44657b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:01.922071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449f445-1d34-4ddd-81c5-9c771e44657b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:01.988119 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1078/2176] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:36:01 2022] POST /identity/v3/projects => generated 376 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:36:01.994902 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f054e-2c71-4cde-a39e-c3b5faff9794 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:02.013887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f054e-2c71-4cde-a39e-c3b5faff9794 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:02.115522 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67b6ff-7ff6-4be6-a9ed-c64c0b941196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:02.116384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67b6ff-7ff6-4be6-a9ed-c64c0b941196 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:02.116716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67b6ff-7ff6-4be6-a9ed-c64c0b941196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:02.117022 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67b6ff-7ff6-4be6-a9ed-c64c0b941196 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:02.117955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67b6ff-7ff6-4be6-a9ed-c64c0b941196 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:02.120909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67b6ff-7ff6-4be6-a9ed-c64c0b941196 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:02.173639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1099/2177] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:36:01 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:36:02.175989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449f445-1d34-4ddd-81c5-9c771e44657b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:02.202508 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449f445-1d34-4ddd-81c5-9c771e44657b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:02.209269 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edcd661-2900-4870-8dab-39dd8e2e72bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:02.210072 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edcd661-2900-4870-8dab-39dd8e2e72bd admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:02.210367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edcd661-2900-4870-8dab-39dd8e2e72bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:02.210744 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edcd661-2900-4870-8dab-39dd8e2e72bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:02.211801 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edcd661-2900-4870-8dab-39dd8e2e72bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:02.216339 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edcd661-2900-4870-8dab-39dd8e2e72bd admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:02.240035 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1079/2178] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:36:02 2022] GET /identity/v3/roles => generated 1427 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:36:02.242763 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67b6ff-7ff6-4be6-a9ed-c64c0b941196 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:02.264816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67b6ff-7ff6-4be6-a9ed-c64c0b941196 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:02.283933 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02928a57-f4a9-4eb3-b5f3-87199ad12f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:02.285483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02928a57-f4a9-4eb3-b5f3-87199ad12f17 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:02.286040 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02928a57-f4a9-4eb3-b5f3-87199ad12f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:02.286508 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02928a57-f4a9-4eb3-b5f3-87199ad12f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:02.287724 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02928a57-f4a9-4eb3-b5f3-87199ad12f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:02.290226 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02928a57-f4a9-4eb3-b5f3-87199ad12f17 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:02.313652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1100/2179] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:36:02 2022] GET /identity/v3/roles => generated 1427 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:36:02.317904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edcd661-2900-4870-8dab-39dd8e2e72bd admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:02.337061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edcd661-2900-4870-8dab-39dd8e2e72bd admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:02.357557 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4cb5f4-12e9-43a2-ae43-57c90276f127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:02.360379 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cb5f4-12e9-43a2-ae43-57c90276f127 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:02.361022 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cb5f4-12e9-43a2-ae43-57c90276f127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:02.361583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cb5f4-12e9-43a2-ae43-57c90276f127 admin admin] PATH_INFO: `/v3/projects/c8e2c1ea53cc4163a7df3474d1498149/users/389ed2f475a3479e821e777b1c4cafb0/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:02.393759 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4cb5f4-12e9-43a2-ae43-57c90276f127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8e2c1ea53cc4163a7df3474d1498149, user_id=389ed2f475a3479e821e777b1c4cafb0, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:02.400848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4cb5f4-12e9-43a2-ae43-57c90276f127 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:02.421543 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1080/2180] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:36:02 2022] PUT /identity/v3/projects/c8e2c1ea53cc4163a7df3474d1498149/users/389ed2f475a3479e821e777b1c4cafb0/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:02.438379 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ec18f-bab5-4737-a9c0-2eca4b5e7474 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:02.438900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ec18f-bab5-4737-a9c0-2eca4b5e7474 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:02.438900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ec18f-bab5-4737-a9c0-2eca4b5e7474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:02.482586 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-813ec18f-bab5-4737-a9c0-2eca4b5e7474 None None] MFA Rules not processed for user `389ed2f475a3479e821e777b1c4cafb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:02.484299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-813ec18f-bab5-4737-a9c0-2eca4b5e7474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:02.599466 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1101/2181] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:02.618742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4cb5f4-12e9-43a2-ae43-57c90276f127 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:02.626681 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ff4cb5f4-12e9-43a2-ae43-57c90276f127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:02.723510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4cb5f4-12e9-43a2-ae43-57c90276f127 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:02.730602 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7b8a2-69f9-4392-96c0-7da7f5f40b30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:02.731326 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7b8a2-69f9-4392-96c0-7da7f5f40b30 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:02.731642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7b8a2-69f9-4392-96c0-7da7f5f40b30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:02.732022 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7b8a2-69f9-4392-96c0-7da7f5f40b30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:02.755351 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b7b8a2-69f9-4392-96c0-7da7f5f40b30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:02.755351 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b7b8a2-69f9-4392-96c0-7da7f5f40b30 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:02.766009 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1081/2182] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:36:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:36:02.786276 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ec18f-bab5-4737-a9c0-2eca4b5e7474 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:02.788019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-813ec18f-bab5-4737-a9c0-2eca4b5e7474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:02.880904 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:02.944371 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ec18f-bab5-4737-a9c0-2eca4b5e7474 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:02.952479 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa18212-3b3e-4ee5-971e-677265701287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:02.953743 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa18212-3b3e-4ee5-971e-677265701287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:02.954415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa18212-3b3e-4ee5-971e-677265701287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:02.954998 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa18212-3b3e-4ee5-971e-677265701287 admin admin] PATH_INFO: `/v3/projects/862c4b20059e4058bdad868151fc2539` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:02.957523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa18212-3b3e-4ee5-971e-677265701287 admin admin] RBAC: Authorizing `identity:delete_project(project_id=862c4b20059e4058bdad868151fc2539)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:02.961616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa18212-3b3e-4ee5-971e-677265701287 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:03.062474 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7fa18212-3b3e-4ee5-971e-677265701287 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '862c4b20059e4058bdad868151fc2539', 'request_id': 'req-7fa18212-3b3e-4ee5-971e-677265701287'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:03.078685 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7fa18212-3b3e-4ee5-971e-677265701287 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '862c4b20059e4058bdad868151fc2539', 'request_id': 'req-7fa18212-3b3e-4ee5-971e-677265701287'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:03.098450 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7fa18212-3b3e-4ee5-971e-677265701287 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '862c4b20059e4058bdad868151fc2539', 'request_id': 'req-7fa18212-3b3e-4ee5-971e-677265701287'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:03.130268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7fa18212-3b3e-4ee5-971e-677265701287 admin admin] The token cache is being invalidate because project 862c4b20059e4058bdad868151fc2539 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:36:03.131071 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7fa18212-3b3e-4ee5-971e-677265701287 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:03.132326 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7fa18212-3b3e-4ee5-971e-677265701287 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:03.137813 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1102/2183] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:36:02 2022] DELETE /identity/v3/projects/862c4b20059e4058bdad868151fc2539 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:03.145249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7b8a2-69f9-4392-96c0-7da7f5f40b30 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:03.173682 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7b8a2-69f9-4392-96c0-7da7f5f40b30 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:03.302689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0902b-68bf-4efb-bdb4-d884979239c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:03.303811 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0902b-68bf-4efb-bdb4-d884979239c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:03.304970 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0902b-68bf-4efb-bdb4-d884979239c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:03.305545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0902b-68bf-4efb-bdb4-d884979239c1 admin admin] PATH_INFO: `/v3/users/8ab110555913420a94a5a3508f48de62` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:03.307894 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db0902b-68bf-4efb-bdb4-d884979239c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ab110555913420a94a5a3508f48de62)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:03.311606 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db0902b-68bf-4efb-bdb4-d884979239c1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:03.323339 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:03.489731 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2db0902b-68bf-4efb-bdb4-d884979239c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ab110555913420a94a5a3508f48de62'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:03.505864 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2db0902b-68bf-4efb-bdb4-d884979239c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ab110555913420a94a5a3508f48de62'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:03.517404 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2db0902b-68bf-4efb-bdb4-d884979239c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ab110555913420a94a5a3508f48de62'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:03.539322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2db0902b-68bf-4efb-bdb4-d884979239c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ab110555913420a94a5a3508f48de62'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:03.541047 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2db0902b-68bf-4efb-bdb4-d884979239c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ab110555913420a94a5a3508f48de62'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:03.569259 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1082/2184] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:36:03 2022] DELETE /identity/v3/users/8ab110555913420a94a5a3508f48de62 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:04.015813 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa18212-3b3e-4ee5-971e-677265701287 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:04.017573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-7fa18212-3b3e-4ee5-971e-677265701287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:04.035083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa18212-3b3e-4ee5-971e-677265701287 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:04.157436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c9974-f1b1-4d97-80c9-874062a23a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:04.158813 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c9974-f1b1-4d97-80c9-874062a23a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:04.159438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c9974-f1b1-4d97-80c9-874062a23a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:04.160327 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c9974-f1b1-4d97-80c9-874062a23a34 admin admin] PATH_INFO: `/v3/projects/bc84aba9a68b4ba08e238101073c6374` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:04.162474 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c9974-f1b1-4d97-80c9-874062a23a34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc84aba9a68b4ba08e238101073c6374)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:04.165325 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c9974-f1b1-4d97-80c9-874062a23a34 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:04.255240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-978c9974-f1b1-4d97-80c9-874062a23a34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc84aba9a68b4ba08e238101073c6374', 'request_id': 'req-978c9974-f1b1-4d97-80c9-874062a23a34'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:04.273378 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-978c9974-f1b1-4d97-80c9-874062a23a34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc84aba9a68b4ba08e238101073c6374', 'request_id': 'req-978c9974-f1b1-4d97-80c9-874062a23a34'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:04.291016 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-978c9974-f1b1-4d97-80c9-874062a23a34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc84aba9a68b4ba08e238101073c6374', 'request_id': 'req-978c9974-f1b1-4d97-80c9-874062a23a34'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:04.293801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:04.308393 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-978c9974-f1b1-4d97-80c9-874062a23a34 admin admin] The token cache is being invalidate because project bc84aba9a68b4ba08e238101073c6374 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:36:04.309036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-978c9974-f1b1-4d97-80c9-874062a23a34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:04.310634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-978c9974-f1b1-4d97-80c9-874062a23a34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:04.312330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1103/2185] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:36:04 2022] DELETE /identity/v3/projects/bc84aba9a68b4ba08e238101073c6374 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:04.317397 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0902b-68bf-4efb-bdb4-d884979239c1 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:04.319055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2db0902b-68bf-4efb-bdb4-d884979239c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:04.333908 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0902b-68bf-4efb-bdb4-d884979239c1 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:04.443585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f420b3-87a2-41fb-8215-bce36206d8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:04.444441 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f420b3-87a2-41fb-8215-bce36206d8b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:04.444968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f420b3-87a2-41fb-8215-bce36206d8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:04.445139 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f420b3-87a2-41fb-8215-bce36206d8b9 admin admin] PATH_INFO: `/v3/users/d62622b727e44c39a23a75478e9ca177` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:04.446365 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f420b3-87a2-41fb-8215-bce36206d8b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d62622b727e44c39a23a75478e9ca177)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:04.448513 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f420b3-87a2-41fb-8215-bce36206d8b9 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:04.543871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b5f420b3-87a2-41fb-8215-bce36206d8b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd62622b727e44c39a23a75478e9ca177'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:04.571114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b5f420b3-87a2-41fb-8215-bce36206d8b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd62622b727e44c39a23a75478e9ca177'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:04.573883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b5f420b3-87a2-41fb-8215-bce36206d8b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd62622b727e44c39a23a75478e9ca177'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:04.622294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b5f420b3-87a2-41fb-8215-bce36206d8b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd62622b727e44c39a23a75478e9ca177'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:04.623567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b5f420b3-87a2-41fb-8215-bce36206d8b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd62622b727e44c39a23a75478e9ca177'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:04.647219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1083/2186] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:36:04 2022] DELETE /identity/v3/users/d62622b727e44c39a23a75478e9ca177 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:05.079870 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c9974-f1b1-4d97-80c9-874062a23a34 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:05.082668 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-978c9974-f1b1-4d97-80c9-874062a23a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:05.104025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c9974-f1b1-4d97-80c9-874062a23a34 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:05.241155 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d1821-e5a4-4ef2-9133-a30101b6a529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:05.242397 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d1821-e5a4-4ef2-9133-a30101b6a529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:05.243109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d1821-e5a4-4ef2-9133-a30101b6a529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:05.243555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d1821-e5a4-4ef2-9133-a30101b6a529 admin admin] PATH_INFO: `/v3/projects/4770e648d6d649798edbaf66702d08f9` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:05.247412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d1821-e5a4-4ef2-9133-a30101b6a529 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4770e648d6d649798edbaf66702d08f9)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:05.251503 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d1821-e5a4-4ef2-9133-a30101b6a529 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:05.346540 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-691d1821-e5a4-4ef2-9133-a30101b6a529 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4770e648d6d649798edbaf66702d08f9', 'request_id': 'req-691d1821-e5a4-4ef2-9133-a30101b6a529'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:05.359296 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-691d1821-e5a4-4ef2-9133-a30101b6a529 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4770e648d6d649798edbaf66702d08f9', 'request_id': 'req-691d1821-e5a4-4ef2-9133-a30101b6a529'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:05.370319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-691d1821-e5a4-4ef2-9133-a30101b6a529 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4770e648d6d649798edbaf66702d08f9', 'request_id': 'req-691d1821-e5a4-4ef2-9133-a30101b6a529'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:05.387909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-691d1821-e5a4-4ef2-9133-a30101b6a529 admin admin] The token cache is being invalidate because project 4770e648d6d649798edbaf66702d08f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:36:05.388310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-691d1821-e5a4-4ef2-9133-a30101b6a529 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:05.389152 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-691d1821-e5a4-4ef2-9133-a30101b6a529 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:05.391260 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1104/2187] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:36:05 2022] DELETE /identity/v3/projects/4770e648d6d649798edbaf66702d08f9 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:05.398108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65679bc2-a0cb-478a-8d5e-cb3a36bfaef1 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:05.398805 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65679bc2-a0cb-478a-8d5e-cb3a36bfaef1 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:05.398805 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65679bc2-a0cb-478a-8d5e-cb3a36bfaef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:05.407677 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:05.433195 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-65679bc2-a0cb-478a-8d5e-cb3a36bfaef1 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:05.434673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-65679bc2-a0cb-478a-8d5e-cb3a36bfaef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:05.552354 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1084/2188] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:05.568114 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d1872-0894-439e-b860-03f541dd18dc None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:05.568750 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d1872-0894-439e-b860-03f541dd18dc None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:05.569259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d1872-0894-439e-b860-03f541dd18dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:05.607722 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-817d1872-0894-439e-b860-03f541dd18dc None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:05.609799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-817d1872-0894-439e-b860-03f541dd18dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:05.635335 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1105/2189] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:05.638390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65679bc2-a0cb-478a-8d5e-cb3a36bfaef1 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:05.663933 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65679bc2-a0cb-478a-8d5e-cb3a36bfaef1 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:05.675569 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f4f3c-ae16-40d5-8ce8-23958a90630d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:05.676378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f4f3c-ae16-40d5-8ce8-23958a90630d admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:05.676691 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f4f3c-ae16-40d5-8ce8-23958a90630d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:05.677020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f4f3c-ae16-40d5-8ce8-23958a90630d admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:05.677969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f4f3c-ae16-40d5-8ce8-23958a90630d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:36:05.678154 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f4f3c-ae16-40d5-8ce8-23958a90630d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:05.679927 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f4f3c-ae16-40d5-8ce8-23958a90630d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:05.700340 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1085/2190] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:36:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:36:05.708027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d1872-0894-439e-b860-03f541dd18dc None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:05.727301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d1872-0894-439e-b860-03f541dd18dc None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:05.745289 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9273a3-c095-4146-9f2b-501bd01c2d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:05.746464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9273a3-c095-4146-9f2b-501bd01c2d6a admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:05.746964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9273a3-c095-4146-9f2b-501bd01c2d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:05.747457 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9273a3-c095-4146-9f2b-501bd01c2d6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:05.748918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9273a3-c095-4146-9f2b-501bd01c2d6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:05.752694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9273a3-c095-4146-9f2b-501bd01c2d6a admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:05.805998 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1106/2191] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:36:05 2022] POST /identity/v3/projects => generated 390 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:36:05.807582 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f4f3c-ae16-40d5-8ce8-23958a90630d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:05.821984 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f4f3c-ae16-40d5-8ce8-23958a90630d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:05.929547 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de2e06-1a1a-4203-b947-ebdf82898d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:05.931663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de2e06-1a1a-4203-b947-ebdf82898d74 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:05.931663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de2e06-1a1a-4203-b947-ebdf82898d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:05.932097 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de2e06-1a1a-4203-b947-ebdf82898d74 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:05.933434 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de2e06-1a1a-4203-b947-ebdf82898d74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:05.937115 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de2e06-1a1a-4203-b947-ebdf82898d74 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:05.980969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1086/2192] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:36:05 2022] POST /identity/v3/users => generated 296 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:36:05.988256 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9273a3-c095-4146-9f2b-501bd01c2d6a admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:06.008471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9273a3-c095-4146-9f2b-501bd01c2d6a admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:06.019616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde11de-d4a3-40a5-9a41-6e3d832da069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:06.020681 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde11de-d4a3-40a5-9a41-6e3d832da069 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:06.021180 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde11de-d4a3-40a5-9a41-6e3d832da069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:06.021896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde11de-d4a3-40a5-9a41-6e3d832da069 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:06.023287 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde11de-d4a3-40a5-9a41-6e3d832da069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:06.026901 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde11de-d4a3-40a5-9a41-6e3d832da069 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:06.043937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1107/2193] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:36:05 2022] GET /identity/v3/roles => generated 1427 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:36:06.049577 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de2e06-1a1a-4203-b947-ebdf82898d74 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:06.067762 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde11de-d4a3-40a5-9a41-6e3d832da069 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:06.071043 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-2cde11de-d4a3-40a5-9a41-6e3d832da069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:06.073074 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de2e06-1a1a-4203-b947-ebdf82898d74 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:06.082542 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668c4cf-9fc9-469a-a8ce-f7c81ed27f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:06.083213 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668c4cf-9fc9-469a-a8ce-f7c81ed27f5f admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:06.083687 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668c4cf-9fc9-469a-a8ce-f7c81ed27f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:06.083814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668c4cf-9fc9-469a-a8ce-f7c81ed27f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:06.084548 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f668c4cf-9fc9-469a-a8ce-f7c81ed27f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:06.086943 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f668c4cf-9fc9-469a-a8ce-f7c81ed27f5f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:06.097441 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde11de-d4a3-40a5-9a41-6e3d832da069 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:06.100975 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1087/2194] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:36:06 2022] GET /identity/v3/roles => generated 1427 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:36:06.107709 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4d0fd-3963-41da-a957-681403e5fffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:06.108985 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4d0fd-3963-41da-a957-681403e5fffa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:06.109487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4d0fd-3963-41da-a957-681403e5fffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:06.109965 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4d0fd-3963-41da-a957-681403e5fffa admin admin] PATH_INFO: `/v3/users/a6734615dd7f4c12bbe2e0acec7e24da` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:06.112441 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4d0fd-3963-41da-a957-681403e5fffa admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6734615dd7f4c12bbe2e0acec7e24da)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:06.114542 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668c4cf-9fc9-469a-a8ce-f7c81ed27f5f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:06.116727 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4d0fd-3963-41da-a957-681403e5fffa admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:06.133032 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668c4cf-9fc9-469a-a8ce-f7c81ed27f5f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:06.142753 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8eab2-6f98-4366-a76c-6fac9171bd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:06.143471 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8eab2-6f98-4366-a76c-6fac9171bd3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:06.144070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8eab2-6f98-4366-a76c-6fac9171bd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:06.144431 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8eab2-6f98-4366-a76c-6fac9171bd3d admin admin] PATH_INFO: `/v3/projects/e0dbc4becf8b4b1388a0bd413b947f78/users/a7a6b7370cf040e3a52e4289b6d081be/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:06.165048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8eab2-6f98-4366-a76c-6fac9171bd3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0dbc4becf8b4b1388a0bd413b947f78, user_id=a7a6b7370cf040e3a52e4289b6d081be, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:06.168648 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8eab2-6f98-4366-a76c-6fac9171bd3d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:06.203785 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1088/2195] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:36:06 2022] PUT /identity/v3/projects/e0dbc4becf8b4b1388a0bd413b947f78/users/a7a6b7370cf040e3a52e4289b6d081be/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:36:06.207126 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:06.214524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373f3d1-1da9-4ae7-8e3a-52efd9fb8ccd None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:06.214524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373f3d1-1da9-4ae7-8e3a-52efd9fb8ccd None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:06.214524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373f3d1-1da9-4ae7-8e3a-52efd9fb8ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:06.225503 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0de4d0fd-3963-41da-a957-681403e5fffa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6734615dd7f4c12bbe2e0acec7e24da'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:06.227983 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0de4d0fd-3963-41da-a957-681403e5fffa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6734615dd7f4c12bbe2e0acec7e24da'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:06.245865 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0de4d0fd-3963-41da-a957-681403e5fffa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6734615dd7f4c12bbe2e0acec7e24da'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:06.251855 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f373f3d1-1da9-4ae7-8e3a-52efd9fb8ccd None None] MFA Rules not processed for user `a7a6b7370cf040e3a52e4289b6d081be`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:06.254053 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f373f3d1-1da9-4ae7-8e3a-52efd9fb8ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:06.258757 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0de4d0fd-3963-41da-a957-681403e5fffa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6734615dd7f4c12bbe2e0acec7e24da'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:06.269192 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0de4d0fd-3963-41da-a957-681403e5fffa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6734615dd7f4c12bbe2e0acec7e24da'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:06.277786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1108/2196] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:36:06 2022] DELETE /identity/v3/users/a6734615dd7f4c12bbe2e0acec7e24da => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:06.283791 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db431b0a-0873-4505-b014-fec8616ec7d3 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:06.284171 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db431b0a-0873-4505-b014-fec8616ec7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:06.284540 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db431b0a-0873-4505-b014-fec8616ec7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:06.307657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:06.309814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-db431b0a-0873-4505-b014-fec8616ec7d3 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:06.311610 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-db431b0a-0873-4505-b014-fec8616ec7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:06.356313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1089/2197] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:06 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:36:06.364049 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373f3d1-1da9-4ae7-8e3a-52efd9fb8ccd None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:06.365545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f373f3d1-1da9-4ae7-8e3a-52efd9fb8ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:06.400017 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1109/2198] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:36:06.414785 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db431b0a-0873-4505-b014-fec8616ec7d3 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:06.416811 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-db431b0a-0873-4505-b014-fec8616ec7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:06.454288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373f3d1-1da9-4ae7-8e3a-52efd9fb8ccd None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:06.460500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609f67c-4a08-431b-bea8-7fdc108ced2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:06.461155 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609f67c-4a08-431b-bea8-7fdc108ced2a admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:06.461486 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609f67c-4a08-431b-bea8-7fdc108ced2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:06.461714 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609f67c-4a08-431b-bea8-7fdc108ced2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:06.462400 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609f67c-4a08-431b-bea8-7fdc108ced2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:06.464764 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609f67c-4a08-431b-bea8-7fdc108ced2a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:06.494854 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db431b0a-0873-4505-b014-fec8616ec7d3 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:06.498624 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1090/2199] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:36:06 2022] POST /identity/v3/projects => generated 390 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:36:06.502122 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92db608-f651-4b19-96a7-ecc0d406ca92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:06.503442 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92db608-f651-4b19-96a7-ecc0d406ca92 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:06.503573 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609f67c-4a08-431b-bea8-7fdc108ced2a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:06.503671 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92db608-f651-4b19-96a7-ecc0d406ca92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:06.504180 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92db608-f651-4b19-96a7-ecc0d406ca92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:06.522674 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92db608-f651-4b19-96a7-ecc0d406ca92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:06.526651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92db608-f651-4b19-96a7-ecc0d406ca92 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:06.540377 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1110/2200] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:36:06 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:36:06.618089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609f67c-4a08-431b-bea8-7fdc108ced2a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:06.624525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e552194-db54-4988-9b6b-1bb6936d9fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:06.625590 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e552194-db54-4988-9b6b-1bb6936d9fc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:06.626074 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e552194-db54-4988-9b6b-1bb6936d9fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:06.627103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e552194-db54-4988-9b6b-1bb6936d9fc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:06.628098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e552194-db54-4988-9b6b-1bb6936d9fc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:06.630385 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e552194-db54-4988-9b6b-1bb6936d9fc2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:06.669666 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1091/2201] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:36:06 2022] POST /identity/v3/users => generated 302 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:36:06.675930 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92db608-f651-4b19-96a7-ecc0d406ca92 service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:06.690956 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92db608-f651-4b19-96a7-ecc0d406ca92 service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:06.700043 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c1cae-395e-4d97-b405-f8110f1e1542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:06.701107 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c1cae-395e-4d97-b405-f8110f1e1542 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:06.701575 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c1cae-395e-4d97-b405-f8110f1e1542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:06.702043 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c1cae-395e-4d97-b405-f8110f1e1542 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:06.703294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096c1cae-395e-4d97-b405-f8110f1e1542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:06.706584 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096c1cae-395e-4d97-b405-f8110f1e1542 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:06.727275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1111/2202] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:36:06 2022] GET /identity/v3/roles => generated 1427 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:36:06.731531 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e552194-db54-4988-9b6b-1bb6936d9fc2 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:06.749855 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e552194-db54-4988-9b6b-1bb6936d9fc2 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:06.756382 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e0de4-3294-4253-817e-c567fbdf522d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:06.757365 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e0de4-3294-4253-817e-c567fbdf522d admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:06.757822 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e0de4-3294-4253-817e-c567fbdf522d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:06.758507 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e0de4-3294-4253-817e-c567fbdf522d admin admin] PATH_INFO: `/v3/projects/9e71172dec2645f79c243f732d85c4ea/users/31b219ababd74e3982403098358385a2/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:06.792765 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e0de4-3294-4253-817e-c567fbdf522d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e71172dec2645f79c243f732d85c4ea, user_id=31b219ababd74e3982403098358385a2, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:06.797070 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e0de4-3294-4253-817e-c567fbdf522d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:06.827681 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1092/2203] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:36:06 2022] PUT /identity/v3/projects/9e71172dec2645f79c243f732d85c4ea/users/31b219ababd74e3982403098358385a2/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:06.833966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c1cae-395e-4d97-b405-f8110f1e1542 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:06.854641 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c1cae-395e-4d97-b405-f8110f1e1542 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:06.918851 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:06.936052 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5753a4a-1a2d-4e28-a4a9-e3effd9ba0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:06.936965 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5753a4a-1a2d-4e28-a4a9-e3effd9ba0cd admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:06.937321 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5753a4a-1a2d-4e28-a4a9-e3effd9ba0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:06.938197 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5753a4a-1a2d-4e28-a4a9-e3effd9ba0cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:06.939021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5753a4a-1a2d-4e28-a4a9-e3effd9ba0cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:06.941451 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5753a4a-1a2d-4e28-a4a9-e3effd9ba0cd admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:06.955070 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1112/2204] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:36:06 2022] GET /identity/v3/roles => generated 1427 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:36:06.960065 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e0de4-3294-4253-817e-c567fbdf522d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:06.975800 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e0de4-3294-4253-817e-c567fbdf522d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:06.982737 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf6483-7b8d-4328-ba8a-8934f50189e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:06.984108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf6483-7b8d-4328-ba8a-8934f50189e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:06.984571 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf6483-7b8d-4328-ba8a-8934f50189e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:06.985016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf6483-7b8d-4328-ba8a-8934f50189e0 admin admin] PATH_INFO: `/v3/projects/9e71172dec2645f79c243f732d85c4ea/users/31b219ababd74e3982403098358385a2/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:06.987837 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edf6483-7b8d-4328-ba8a-8934f50189e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e71172dec2645f79c243f732d85c4ea, user_id=31b219ababd74e3982403098358385a2, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:06.991505 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edf6483-7b8d-4328-ba8a-8934f50189e0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:07.005179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1093/2205] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:36:06 2022] PUT /identity/v3/projects/9e71172dec2645f79c243f732d85c4ea/users/31b219ababd74e3982403098358385a2/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:07.018607 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136863d-e186-450d-8d45-e55f1547a0b2 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:07.019200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136863d-e186-450d-8d45-e55f1547a0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:07.019741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136863d-e186-450d-8d45-e55f1547a0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:07.069153 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-2136863d-e186-450d-8d45-e55f1547a0b2 None None] MFA Rules not processed for user `31b219ababd74e3982403098358385a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:07.070596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-2136863d-e186-450d-8d45-e55f1547a0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:07.087152 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf6483-7b8d-4328-ba8a-8934f50189e0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:07.089813 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3edf6483-7b8d-4328-ba8a-8934f50189e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:07.123541 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:07.203636 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1113/2206] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:07 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:36:07.211744 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf6483-7b8d-4328-ba8a-8934f50189e0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:07.217778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136863d-e186-450d-8d45-e55f1547a0b2 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:07.218177 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557dfe5d-7986-4b5e-9da3-7b633654a0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:07.218727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557dfe5d-7986-4b5e-9da3-7b633654a0e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:07.218727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557dfe5d-7986-4b5e-9da3-7b633654a0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:07.219859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557dfe5d-7986-4b5e-9da3-7b633654a0e4 admin admin] PATH_INFO: `/v3/projects/891d22474d19456ab70b881060dadde0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:07.221096 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-2136863d-e186-450d-8d45-e55f1547a0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:07.222044 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557dfe5d-7986-4b5e-9da3-7b633654a0e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=891d22474d19456ab70b881060dadde0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:07.226992 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557dfe5d-7986-4b5e-9da3-7b633654a0e4 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:07.306877 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136863d-e186-450d-8d45-e55f1547a0b2 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:07.311786 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-557dfe5d-7986-4b5e-9da3-7b633654a0e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '891d22474d19456ab70b881060dadde0', 'request_id': 'req-557dfe5d-7986-4b5e-9da3-7b633654a0e4'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:07.324026 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-557dfe5d-7986-4b5e-9da3-7b633654a0e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '891d22474d19456ab70b881060dadde0', 'request_id': 'req-557dfe5d-7986-4b5e-9da3-7b633654a0e4'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:07.331855 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-557dfe5d-7986-4b5e-9da3-7b633654a0e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '891d22474d19456ab70b881060dadde0', 'request_id': 'req-557dfe5d-7986-4b5e-9da3-7b633654a0e4'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:07.350144 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-557dfe5d-7986-4b5e-9da3-7b633654a0e4 admin admin] The token cache is being invalidate because project 891d22474d19456ab70b881060dadde0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:36:07.350467 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-557dfe5d-7986-4b5e-9da3-7b633654a0e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:07.351194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-557dfe5d-7986-4b5e-9da3-7b633654a0e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:07.353353 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1094/2207] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:36:07 2022] DELETE /identity/v3/projects/891d22474d19456ab70b881060dadde0 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:07.397880 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e7e62-0b22-45ba-ad03-9bb7e7a2b732 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:07.398739 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e7e62-0b22-45ba-ad03-9bb7e7a2b732 service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:07.399059 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e7e62-0b22-45ba-ad03-9bb7e7a2b732 service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:07.399385 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e7e62-0b22-45ba-ad03-9bb7e7a2b732 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:07.402084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-ca9e7e62-0b22-45ba-ad03-9bb7e7a2b732 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:07.482174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:07.518615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e7e62-0b22-45ba-ad03-9bb7e7a2b732 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:07.520600 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e7e62-0b22-45ba-ad03-9bb7e7a2b732 service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:07.533318 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1114/2208] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:36:07 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:36:07.724111 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557dfe5d-7986-4b5e-9da3-7b633654a0e4 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:07.725894 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-557dfe5d-7986-4b5e-9da3-7b633654a0e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:07.828437 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557dfe5d-7986-4b5e-9da3-7b633654a0e4 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:07.836852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a767472-f8af-4632-bf3b-c17c0e717e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:07.837915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a767472-f8af-4632-bf3b-c17c0e717e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:07.838370 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a767472-f8af-4632-bf3b-c17c0e717e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:07.838824 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a767472-f8af-4632-bf3b-c17c0e717e97 admin admin] PATH_INFO: `/v3/users/a7a6b7370cf040e3a52e4289b6d081be` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:07.840834 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a767472-f8af-4632-bf3b-c17c0e717e97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7a6b7370cf040e3a52e4289b6d081be)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:07.844364 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a767472-f8af-4632-bf3b-c17c0e717e97 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:07.933071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:07.935925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5a767472-f8af-4632-bf3b-c17c0e717e97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7a6b7370cf040e3a52e4289b6d081be'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:07.951862 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5a767472-f8af-4632-bf3b-c17c0e717e97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7a6b7370cf040e3a52e4289b6d081be'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:07.953157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5a767472-f8af-4632-bf3b-c17c0e717e97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7a6b7370cf040e3a52e4289b6d081be'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:07.971995 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5a767472-f8af-4632-bf3b-c17c0e717e97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7a6b7370cf040e3a52e4289b6d081be'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:07.973152 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5a767472-f8af-4632-bf3b-c17c0e717e97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7a6b7370cf040e3a52e4289b6d081be'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:08.002636 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1095/2209] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:36:07 2022] DELETE /identity/v3/users/a7a6b7370cf040e3a52e4289b6d081be => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:08.009202 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e68e01-d899-494e-a014-34e2ce5fb257 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:08.009585 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e68e01-d899-494e-a014-34e2ce5fb257 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:08.009917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e68e01-d899-494e-a014-34e2ce5fb257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:08.032976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-c1e68e01-d899-494e-a014-34e2ce5fb257 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:08.034474 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c1e68e01-d899-494e-a014-34e2ce5fb257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:08.114095 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1115/2210] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:08.125630 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a767472-f8af-4632-bf3b-c17c0e717e97 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:08.127798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5a767472-f8af-4632-bf3b-c17c0e717e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:08.222487 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a767472-f8af-4632-bf3b-c17c0e717e97 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:08.226453 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c6a65-47ca-471f-bde3-4adf2bf80285 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:08.228146 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c6a65-47ca-471f-bde3-4adf2bf80285 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:08.228146 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c6a65-47ca-471f-bde3-4adf2bf80285 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:08.228146 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c6a65-47ca-471f-bde3-4adf2bf80285 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:08.238905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c6a65-47ca-471f-bde3-4adf2bf80285 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:08.240458 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c6a65-47ca-471f-bde3-4adf2bf80285 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:08.254870 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1096/2211] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:36:08 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:36:08.631626 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e68e01-d899-494e-a014-34e2ce5fb257 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:08.636751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c1e68e01-d899-494e-a014-34e2ce5fb257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:08.660983 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e68e01-d899-494e-a014-34e2ce5fb257 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:08.668998 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181f110-d472-4aa9-918e-160b87be0a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:08.669697 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181f110-d472-4aa9-918e-160b87be0a60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:08.669997 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181f110-d472-4aa9-918e-160b87be0a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:08.670265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181f110-d472-4aa9-918e-160b87be0a60 admin admin] PATH_INFO: `/v3/projects/e0dbc4becf8b4b1388a0bd413b947f78` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:08.671820 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181f110-d472-4aa9-918e-160b87be0a60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0dbc4becf8b4b1388a0bd413b947f78)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:08.674042 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181f110-d472-4aa9-918e-160b87be0a60 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:08.764809 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b181f110-d472-4aa9-918e-160b87be0a60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0dbc4becf8b4b1388a0bd413b947f78', 'request_id': 'req-b181f110-d472-4aa9-918e-160b87be0a60'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:08.775198 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b181f110-d472-4aa9-918e-160b87be0a60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0dbc4becf8b4b1388a0bd413b947f78', 'request_id': 'req-b181f110-d472-4aa9-918e-160b87be0a60'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:08.785855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b181f110-d472-4aa9-918e-160b87be0a60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0dbc4becf8b4b1388a0bd413b947f78', 'request_id': 'req-b181f110-d472-4aa9-918e-160b87be0a60'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:08.810955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b181f110-d472-4aa9-918e-160b87be0a60 admin admin] The token cache is being invalidate because project e0dbc4becf8b4b1388a0bd413b947f78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:36:08.811615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b181f110-d472-4aa9-918e-160b87be0a60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:08.813571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b181f110-d472-4aa9-918e-160b87be0a60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:08.816045 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1116/2212] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:36:08 2022] DELETE /identity/v3/projects/e0dbc4becf8b4b1388a0bd413b947f78 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:08.827909 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c6a65-47ca-471f-bde3-4adf2bf80285 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:08.842073 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c6a65-47ca-471f-bde3-4adf2bf80285 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:08.938268 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd123a3b-9ebb-4b6a-83df-54b6009fc656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:08.939635 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd123a3b-9ebb-4b6a-83df-54b6009fc656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:08.940128 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd123a3b-9ebb-4b6a-83df-54b6009fc656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:08.940593 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd123a3b-9ebb-4b6a-83df-54b6009fc656 admin admin] PATH_INFO: `/v3/users/31b219ababd74e3982403098358385a2` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:08.942705 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd123a3b-9ebb-4b6a-83df-54b6009fc656 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31b219ababd74e3982403098358385a2)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:08.946512 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd123a3b-9ebb-4b6a-83df-54b6009fc656 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:09.018332 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fd123a3b-9ebb-4b6a-83df-54b6009fc656 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31b219ababd74e3982403098358385a2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:09.027116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fd123a3b-9ebb-4b6a-83df-54b6009fc656 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31b219ababd74e3982403098358385a2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:09.027978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fd123a3b-9ebb-4b6a-83df-54b6009fc656 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31b219ababd74e3982403098358385a2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:09.035249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:09.043276 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fd123a3b-9ebb-4b6a-83df-54b6009fc656 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31b219ababd74e3982403098358385a2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:09.044872 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fd123a3b-9ebb-4b6a-83df-54b6009fc656 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31b219ababd74e3982403098358385a2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:09.066100 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1097/2213] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:36:08 2022] DELETE /identity/v3/users/31b219ababd74e3982403098358385a2 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:09.581402 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181f110-d472-4aa9-918e-160b87be0a60 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:09.587034 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-b181f110-d472-4aa9-918e-160b87be0a60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:09.618608 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181f110-d472-4aa9-918e-160b87be0a60 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:09.698978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a0d27-3f0e-4b70-ac0d-37d136d53702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:09.700319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a0d27-3f0e-4b70-ac0d-37d136d53702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:09.700319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a0d27-3f0e-4b70-ac0d-37d136d53702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:09.700319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a0d27-3f0e-4b70-ac0d-37d136d53702 admin admin] PATH_INFO: `/v3/projects/9e71172dec2645f79c243f732d85c4ea` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:09.701800 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a0d27-3f0e-4b70-ac0d-37d136d53702 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e71172dec2645f79c243f732d85c4ea)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:09.703998 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a0d27-3f0e-4b70-ac0d-37d136d53702 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:09.707752 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:09.778443 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-755a0d27-3f0e-4b70-ac0d-37d136d53702 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e71172dec2645f79c243f732d85c4ea', 'request_id': 'req-755a0d27-3f0e-4b70-ac0d-37d136d53702'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:09.791432 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-755a0d27-3f0e-4b70-ac0d-37d136d53702 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e71172dec2645f79c243f732d85c4ea', 'request_id': 'req-755a0d27-3f0e-4b70-ac0d-37d136d53702'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:09.801290 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-755a0d27-3f0e-4b70-ac0d-37d136d53702 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e71172dec2645f79c243f732d85c4ea', 'request_id': 'req-755a0d27-3f0e-4b70-ac0d-37d136d53702'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:09.829806 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-755a0d27-3f0e-4b70-ac0d-37d136d53702 admin admin] The token cache is being invalidate because project 9e71172dec2645f79c243f732d85c4ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:36:09.830302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-755a0d27-3f0e-4b70-ac0d-37d136d53702 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:09.831726 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-755a0d27-3f0e-4b70-ac0d-37d136d53702 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:09.840298 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1117/2214] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:36:09 2022] DELETE /identity/v3/projects/9e71172dec2645f79c243f732d85c4ea => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:09.844779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2df351-90e9-40de-84bb-b6d45b9ff3d4 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:09.848144 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2df351-90e9-40de-84bb-b6d45b9ff3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:09.848681 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2df351-90e9-40de-84bb-b6d45b9ff3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:09.878651 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-cd2df351-90e9-40de-84bb-b6d45b9ff3d4 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:09.880592 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cd2df351-90e9-40de-84bb-b6d45b9ff3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:09.998588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1098/2215] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:09 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:10.001118 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658896ee-08a5-4b6f-bccd-6392085d5773 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:10.001481 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658896ee-08a5-4b6f-bccd-6392085d5773 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:10.001839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658896ee-08a5-4b6f-bccd-6392085d5773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:10.036220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-658896ee-08a5-4b6f-bccd-6392085d5773 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:10.037743 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-658896ee-08a5-4b6f-bccd-6392085d5773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:10.048131 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1118/2216] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:09 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:10.053367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2df351-90e9-40de-84bb-b6d45b9ff3d4 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:10.064144 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2df351-90e9-40de-84bb-b6d45b9ff3d4 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:10.081973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e34e7a-4aa9-4d6e-a67f-daad9b892d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:10.082677 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e34e7a-4aa9-4d6e-a67f-daad9b892d22 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:10.082973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e34e7a-4aa9-4d6e-a67f-daad9b892d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:10.083249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e34e7a-4aa9-4d6e-a67f-daad9b892d22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:10.084328 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e34e7a-4aa9-4d6e-a67f-daad9b892d22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:36:10.084870 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e34e7a-4aa9-4d6e-a67f-daad9b892d22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:10.088124 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e34e7a-4aa9-4d6e-a67f-daad9b892d22 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:10.110658 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1099/2217] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:36:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:36:10.115712 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658896ee-08a5-4b6f-bccd-6392085d5773 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:10.126387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658896ee-08a5-4b6f-bccd-6392085d5773 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:10.132188 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697549e4-195d-404d-8645-34e8d785bbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:10.132890 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697549e4-195d-404d-8645-34e8d785bbd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:10.133181 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697549e4-195d-404d-8645-34e8d785bbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:10.133464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697549e4-195d-404d-8645-34e8d785bbd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:10.134199 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697549e4-195d-404d-8645-34e8d785bbd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:10.136266 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697549e4-195d-404d-8645-34e8d785bbd6 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:10.171034 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1119/2218] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:36:10 2022] POST /identity/v3/projects => generated 386 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:36:10.177835 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e34e7a-4aa9-4d6e-a67f-daad9b892d22 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:10.190365 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e34e7a-4aa9-4d6e-a67f-daad9b892d22 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:10.261813 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e7bf1-ce2f-4ce8-a6c6-586eb7688fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:10.262557 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e7bf1-ce2f-4ce8-a6c6-586eb7688fd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:10.262964 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e7bf1-ce2f-4ce8-a6c6-586eb7688fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:10.263257 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e7bf1-ce2f-4ce8-a6c6-586eb7688fd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:10.264188 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e7bf1-ce2f-4ce8-a6c6-586eb7688fd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:10.267040 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e7bf1-ce2f-4ce8-a6c6-586eb7688fd3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:10.293272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1100/2219] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:36:10 2022] POST /identity/v3/users => generated 294 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:36:10.299164 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697549e4-195d-404d-8645-34e8d785bbd6 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:10.310881 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697549e4-195d-404d-8645-34e8d785bbd6 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:10.316994 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bbcd73-a884-4314-809c-1ea107cb81d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:10.317745 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bbcd73-a884-4314-809c-1ea107cb81d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:10.318058 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bbcd73-a884-4314-809c-1ea107cb81d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:10.318402 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bbcd73-a884-4314-809c-1ea107cb81d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:10.319414 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bbcd73-a884-4314-809c-1ea107cb81d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:10.321144 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bbcd73-a884-4314-809c-1ea107cb81d3 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:10.333995 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1120/2220] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:36:10 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:36:10.338771 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e7bf1-ce2f-4ce8-a6c6-586eb7688fd3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:10.350901 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e7bf1-ce2f-4ce8-a6c6-586eb7688fd3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:10.356423 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af321ed-f668-4dfb-80b5-48f440ff3b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:10.357096 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af321ed-f668-4dfb-80b5-48f440ff3b31 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:10.357384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af321ed-f668-4dfb-80b5-48f440ff3b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:10.357667 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af321ed-f668-4dfb-80b5-48f440ff3b31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:10.358470 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af321ed-f668-4dfb-80b5-48f440ff3b31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:10.360164 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af321ed-f668-4dfb-80b5-48f440ff3b31 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:10.372722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1101/2221] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:36:10 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:36:10.377431 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bbcd73-a884-4314-809c-1ea107cb81d3 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:10.389632 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bbcd73-a884-4314-809c-1ea107cb81d3 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:10.395365 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09483434-cf87-4111-95c4-b23d455d5cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:10.396376 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09483434-cf87-4111-95c4-b23d455d5cfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:10.396801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09483434-cf87-4111-95c4-b23d455d5cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:10.397084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09483434-cf87-4111-95c4-b23d455d5cfe admin admin] PATH_INFO: `/v3/projects/597e5f452b6a459889532631d8a13f33/users/be04efee895048b9809c3fc216677f95/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:10.422300 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09483434-cf87-4111-95c4-b23d455d5cfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=597e5f452b6a459889532631d8a13f33, user_id=be04efee895048b9809c3fc216677f95, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:10.428303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09483434-cf87-4111-95c4-b23d455d5cfe admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:10.446894 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1121/2222] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:36:10 2022] PUT /identity/v3/projects/597e5f452b6a459889532631d8a13f33/users/be04efee895048b9809c3fc216677f95/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:10.458861 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b323044-21cd-4f80-9f2f-b85ec32c1154 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:10.459155 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b323044-21cd-4f80-9f2f-b85ec32c1154 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:10.459694 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b323044-21cd-4f80-9f2f-b85ec32c1154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:10.493746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8b323044-21cd-4f80-9f2f-b85ec32c1154 None None] MFA Rules not processed for user `be04efee895048b9809c3fc216677f95`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:10.495069 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8b323044-21cd-4f80-9f2f-b85ec32c1154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:10.550762 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:10.612846 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1102/2223] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:10 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:10.618790 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09483434-cf87-4111-95c4-b23d455d5cfe admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:10.696245 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09483434-cf87-4111-95c4-b23d455d5cfe admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:10.701469 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d40732-14b6-4f56-8852-187af9359199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:10.702198 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d40732-14b6-4f56-8852-187af9359199 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:10.702402 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d40732-14b6-4f56-8852-187af9359199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:10.702681 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d40732-14b6-4f56-8852-187af9359199 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:10.703533 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d40732-14b6-4f56-8852-187af9359199 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:10.705485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d40732-14b6-4f56-8852-187af9359199 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:10.736373 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1122/2224] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:36:10 2022] POST /identity/v3/projects => generated 386 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:36:10.741163 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b323044-21cd-4f80-9f2f-b85ec32c1154 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:10.753362 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b323044-21cd-4f80-9f2f-b85ec32c1154 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:10.824271 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a028d-e5cf-4c8c-a119-a749c56f721b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:10.825313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a028d-e5cf-4c8c-a119-a749c56f721b admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:10.825761 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a028d-e5cf-4c8c-a119-a749c56f721b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:10.826030 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a028d-e5cf-4c8c-a119-a749c56f721b admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:10.826889 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a028d-e5cf-4c8c-a119-a749c56f721b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:10.829204 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a028d-e5cf-4c8c-a119-a749c56f721b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:10.861287 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1103/2225] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:36:10 2022] POST /identity/v3/users => generated 300 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:36:10.866624 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d40732-14b6-4f56-8852-187af9359199 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:10.884689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d40732-14b6-4f56-8852-187af9359199 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:10.890901 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6a708-1cb4-4168-8332-276e3d83efcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:10.891561 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6a708-1cb4-4168-8332-276e3d83efcb admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:10.891845 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6a708-1cb4-4168-8332-276e3d83efcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:10.892216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6a708-1cb4-4168-8332-276e3d83efcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:10.892994 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b6a708-1cb4-4168-8332-276e3d83efcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:10.894908 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b6a708-1cb4-4168-8332-276e3d83efcb admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:10.916897 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a028d-e5cf-4c8c-a119-a749c56f721b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:10.918280 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1123/2226] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:36:10 2022] GET /identity/v3/roles => generated 1427 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:36:10.935920 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a028d-e5cf-4c8c-a119-a749c56f721b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:10.947707 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e51d3-5f93-419d-872e-cc7d726aeb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:10.950175 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e51d3-5f93-419d-872e-cc7d726aeb4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:10.950469 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e51d3-5f93-419d-872e-cc7d726aeb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:10.951023 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e51d3-5f93-419d-872e-cc7d726aeb4f admin admin] PATH_INFO: `/v3/projects/a9a11895d18e4eccabcad782323f8936/users/93a315b24c9141948d2319605ede5b0a/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:10.973144 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e51d3-5f93-419d-872e-cc7d726aeb4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9a11895d18e4eccabcad782323f8936, user_id=93a315b24c9141948d2319605ede5b0a, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:10.976357 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e51d3-5f93-419d-872e-cc7d726aeb4f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:11.008032 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1104/2227] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:36:10 2022] PUT /identity/v3/projects/a9a11895d18e4eccabcad782323f8936/users/93a315b24c9141948d2319605ede5b0a/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:11.015498 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6a708-1cb4-4168-8332-276e3d83efcb admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:11.022046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:11.033986 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6a708-1cb4-4168-8332-276e3d83efcb admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:11.126962 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7897058-47f5-4b42-90b0-51086ee3ed27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:11.127630 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7897058-47f5-4b42-90b0-51086ee3ed27 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:11.127925 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7897058-47f5-4b42-90b0-51086ee3ed27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:11.128304 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7897058-47f5-4b42-90b0-51086ee3ed27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:11.129398 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7897058-47f5-4b42-90b0-51086ee3ed27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:11.131117 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7897058-47f5-4b42-90b0-51086ee3ed27 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:11.158035 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1124/2228] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:36:11 2022] GET /identity/v3/roles => generated 1427 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:36:11.158537 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e51d3-5f93-419d-872e-cc7d726aeb4f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:11.161756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e51d3-5f93-419d-872e-cc7d726aeb4f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:11.167925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2da8e2-29f7-43d5-b907-ffc47d89ce25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:11.168782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2da8e2-29f7-43d5-b907-ffc47d89ce25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:11.169170 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2da8e2-29f7-43d5-b907-ffc47d89ce25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:11.169555 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2da8e2-29f7-43d5-b907-ffc47d89ce25 admin admin] PATH_INFO: `/v3/projects/a9a11895d18e4eccabcad782323f8936/users/93a315b24c9141948d2319605ede5b0a/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:11.172233 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2da8e2-29f7-43d5-b907-ffc47d89ce25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9a11895d18e4eccabcad782323f8936, user_id=93a315b24c9141948d2319605ede5b0a, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:11.176086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2da8e2-29f7-43d5-b907-ffc47d89ce25 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:11.193225 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1105/2229] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:36:11 2022] PUT /identity/v3/projects/a9a11895d18e4eccabcad782323f8936/users/93a315b24c9141948d2319605ede5b0a/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:11.206366 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edb194-2c4d-4250-be97-1c0baaaeac48 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:11.206948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edb194-2c4d-4250-be97-1c0baaaeac48 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:11.207306 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edb194-2c4d-4250-be97-1c0baaaeac48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:11.248760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-e9edb194-2c4d-4250-be97-1c0baaaeac48 None None] MFA Rules not processed for user `93a315b24c9141948d2319605ede5b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:11.250579 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-e9edb194-2c4d-4250-be97-1c0baaaeac48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:11.424684 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1125/2230] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:11 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:11.438277 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2da8e2-29f7-43d5-b907-ffc47d89ce25 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:11.439907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7a2da8e2-29f7-43d5-b907-ffc47d89ce25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:11.512526 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2da8e2-29f7-43d5-b907-ffc47d89ce25 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:11.516388 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181f6d2-4e3e-485e-9326-c799a1a4984a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:11.517007 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181f6d2-4e3e-485e-9326-c799a1a4984a service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:11.517362 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181f6d2-4e3e-485e-9326-c799a1a4984a service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:11.517591 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181f6d2-4e3e-485e-9326-c799a1a4984a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:11.528256 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181f6d2-4e3e-485e-9326-c799a1a4984a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:11.529790 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181f6d2-4e3e-485e-9326-c799a1a4984a service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:11.541074 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1106/2231] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:36:11 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:36:12.452675 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9edb194-2c4d-4250-be97-1c0baaaeac48 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:12.519840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9edb194-2c4d-4250-be97-1c0baaaeac48 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:12.524888 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba03aa-632f-4b9f-96ec-42caa7a3d988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:12.525586 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba03aa-632f-4b9f-96ec-42caa7a3d988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:12.525869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba03aa-632f-4b9f-96ec-42caa7a3d988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:12.526153 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba03aa-632f-4b9f-96ec-42caa7a3d988 admin admin] PATH_INFO: `/v3/users/be04efee895048b9809c3fc216677f95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:12.527477 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba03aa-632f-4b9f-96ec-42caa7a3d988 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be04efee895048b9809c3fc216677f95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:12.529551 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba03aa-632f-4b9f-96ec-42caa7a3d988 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:12.569593 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:12.595902 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-80ba03aa-632f-4b9f-96ec-42caa7a3d988 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be04efee895048b9809c3fc216677f95'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:12.596722 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-80ba03aa-632f-4b9f-96ec-42caa7a3d988 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be04efee895048b9809c3fc216677f95'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:12.612895 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-80ba03aa-632f-4b9f-96ec-42caa7a3d988 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be04efee895048b9809c3fc216677f95'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:12.620697 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-80ba03aa-632f-4b9f-96ec-42caa7a3d988 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be04efee895048b9809c3fc216677f95'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:12.630943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-80ba03aa-632f-4b9f-96ec-42caa7a3d988 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be04efee895048b9809c3fc216677f95'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:12.638142 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1126/2232] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:36:12 2022] DELETE /identity/v3/users/be04efee895048b9809c3fc216677f95 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:12.647916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666ad25-40d0-4f48-94cd-2f7bf3c37f54 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:12.648902 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666ad25-40d0-4f48-94cd-2f7bf3c37f54 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:12.649217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666ad25-40d0-4f48-94cd-2f7bf3c37f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:12.671368 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-1666ad25-40d0-4f48-94cd-2f7bf3c37f54 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:12.672818 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1666ad25-40d0-4f48-94cd-2f7bf3c37f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:12.713333 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:12.750817 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1107/2233] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:12 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:12.761619 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba03aa-632f-4b9f-96ec-42caa7a3d988 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:12.763292 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-80ba03aa-632f-4b9f-96ec-42caa7a3d988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:12.834649 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba03aa-632f-4b9f-96ec-42caa7a3d988 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:12.839008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414eea4-56a8-425d-8ccb-264a6f66b47b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:12.839659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414eea4-56a8-425d-8ccb-264a6f66b47b service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:12.839978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414eea4-56a8-425d-8ccb-264a6f66b47b service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:12.840274 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414eea4-56a8-425d-8ccb-264a6f66b47b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:12.853796 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414eea4-56a8-425d-8ccb-264a6f66b47b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:12.855444 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414eea4-56a8-425d-8ccb-264a6f66b47b service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:12.880458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1127/2234] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:36:12 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:36:13.267569 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666ad25-40d0-4f48-94cd-2f7bf3c37f54 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:13.273702 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1666ad25-40d0-4f48-94cd-2f7bf3c37f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:13.294403 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666ad25-40d0-4f48-94cd-2f7bf3c37f54 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:13.301431 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508cc41a-1ea9-46b4-9764-bc592adc33ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:13.302654 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cc41a-1ea9-46b4-9764-bc592adc33ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:13.303118 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cc41a-1ea9-46b4-9764-bc592adc33ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:13.303555 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cc41a-1ea9-46b4-9764-bc592adc33ea admin admin] PATH_INFO: `/v3/projects/597e5f452b6a459889532631d8a13f33` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:13.310464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508cc41a-1ea9-46b4-9764-bc592adc33ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=597e5f452b6a459889532631d8a13f33)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:13.312935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508cc41a-1ea9-46b4-9764-bc592adc33ea admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:13.401649 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-508cc41a-1ea9-46b4-9764-bc592adc33ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '597e5f452b6a459889532631d8a13f33', 'request_id': 'req-508cc41a-1ea9-46b4-9764-bc592adc33ea'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:13.409496 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-508cc41a-1ea9-46b4-9764-bc592adc33ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '597e5f452b6a459889532631d8a13f33', 'request_id': 'req-508cc41a-1ea9-46b4-9764-bc592adc33ea'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:13.418364 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-508cc41a-1ea9-46b4-9764-bc592adc33ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '597e5f452b6a459889532631d8a13f33', 'request_id': 'req-508cc41a-1ea9-46b4-9764-bc592adc33ea'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:13.436012 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-508cc41a-1ea9-46b4-9764-bc592adc33ea admin admin] The token cache is being invalidate because project 597e5f452b6a459889532631d8a13f33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:36:13.436447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-508cc41a-1ea9-46b4-9764-bc592adc33ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:13.437220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-508cc41a-1ea9-46b4-9764-bc592adc33ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:13.439409 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1108/2235] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:36:13 2022] DELETE /identity/v3/projects/597e5f452b6a459889532631d8a13f33 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:13.444427 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414eea4-56a8-425d-8ccb-264a6f66b47b service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:13.455565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414eea4-56a8-425d-8ccb-264a6f66b47b service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:13.533985 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99e9c2-bc3e-47f4-b10b-0fc1403c440e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:13.533985 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99e9c2-bc3e-47f4-b10b-0fc1403c440e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:13.534244 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99e9c2-bc3e-47f4-b10b-0fc1403c440e admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:13.534485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99e9c2-bc3e-47f4-b10b-0fc1403c440e admin admin] PATH_INFO: `/v3/users/93a315b24c9141948d2319605ede5b0a` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:13.535753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99e9c2-bc3e-47f4-b10b-0fc1403c440e admin admin] RBAC: Authorizing `identity:delete_user(user_id=93a315b24c9141948d2319605ede5b0a)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:13.537973 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99e9c2-bc3e-47f4-b10b-0fc1403c440e admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:13.629654 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-2e99e9c2-bc3e-47f4-b10b-0fc1403c440e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93a315b24c9141948d2319605ede5b0a'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:13.631141 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-2e99e9c2-bc3e-47f4-b10b-0fc1403c440e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93a315b24c9141948d2319605ede5b0a'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:13.632885 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:13.660953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-2e99e9c2-bc3e-47f4-b10b-0fc1403c440e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93a315b24c9141948d2319605ede5b0a'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:13.674725 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-2e99e9c2-bc3e-47f4-b10b-0fc1403c440e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93a315b24c9141948d2319605ede5b0a'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:13.685815 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-2e99e9c2-bc3e-47f4-b10b-0fc1403c440e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93a315b24c9141948d2319605ede5b0a'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:13.699139 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1128/2236] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:36:13 2022] DELETE /identity/v3/users/93a315b24c9141948d2319605ede5b0a => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:14.055077 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508cc41a-1ea9-46b4-9764-bc592adc33ea admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:14.056884 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-508cc41a-1ea9-46b4-9764-bc592adc33ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:14.076271 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508cc41a-1ea9-46b4-9764-bc592adc33ea admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:14.153558 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58236d54-07ac-431e-a9d5-a6bc829b6685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:14.154023 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58236d54-07ac-431e-a9d5-a6bc829b6685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:14.154321 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58236d54-07ac-431e-a9d5-a6bc829b6685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:14.154606 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58236d54-07ac-431e-a9d5-a6bc829b6685 admin admin] PATH_INFO: `/v3/projects/a9a11895d18e4eccabcad782323f8936` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:14.161686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58236d54-07ac-431e-a9d5-a6bc829b6685 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9a11895d18e4eccabcad782323f8936)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:14.164044 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58236d54-07ac-431e-a9d5-a6bc829b6685 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:14.230480 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-58236d54-07ac-431e-a9d5-a6bc829b6685 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9a11895d18e4eccabcad782323f8936', 'request_id': 'req-58236d54-07ac-431e-a9d5-a6bc829b6685'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:14.240325 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-58236d54-07ac-431e-a9d5-a6bc829b6685 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9a11895d18e4eccabcad782323f8936', 'request_id': 'req-58236d54-07ac-431e-a9d5-a6bc829b6685'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:14.246153 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:14.251660 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-58236d54-07ac-431e-a9d5-a6bc829b6685 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9a11895d18e4eccabcad782323f8936', 'request_id': 'req-58236d54-07ac-431e-a9d5-a6bc829b6685'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:14.272001 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-58236d54-07ac-431e-a9d5-a6bc829b6685 admin admin] The token cache is being invalidate because project a9a11895d18e4eccabcad782323f8936 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:36:14.272460 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-58236d54-07ac-431e-a9d5-a6bc829b6685 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:14.274204 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-58236d54-07ac-431e-a9d5-a6bc829b6685 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:14.281861 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1109/2237] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:36:14 2022] DELETE /identity/v3/projects/a9a11895d18e4eccabcad782323f8936 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:14.283873 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021f0db-7eb0-475c-9781-0a30f0aac008 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:14.284336 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021f0db-7eb0-475c-9781-0a30f0aac008 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:14.284489 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021f0db-7eb0-475c-9781-0a30f0aac008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:14.306536 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-6021f0db-7eb0-475c-9781-0a30f0aac008 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:14.307916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6021f0db-7eb0-475c-9781-0a30f0aac008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:14.413976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1129/2238] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:14.423465 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ed475-d8f3-49bb-bc38-001fd866a0dc None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:14.423817 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ed475-d8f3-49bb-bc38-001fd866a0dc None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:14.424538 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ed475-d8f3-49bb-bc38-001fd866a0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:14.450372 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f15ed475-d8f3-49bb-bc38-001fd866a0dc None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:14.451637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f15ed475-d8f3-49bb-bc38-001fd866a0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:14.462038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1110/2239] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:14.467032 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021f0db-7eb0-475c-9781-0a30f0aac008 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:14.480186 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021f0db-7eb0-475c-9781-0a30f0aac008 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:14.492529 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f4c52-a612-4a7e-b48f-4522d016871b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:14.493315 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f4c52-a612-4a7e-b48f-4522d016871b admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:14.493724 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f4c52-a612-4a7e-b48f-4522d016871b admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:14.494214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f4c52-a612-4a7e-b48f-4522d016871b admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:14.495049 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f4c52-a612-4a7e-b48f-4522d016871b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:36:14.495523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f4c52-a612-4a7e-b48f-4522d016871b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:14.497430 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f4c52-a612-4a7e-b48f-4522d016871b admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:14.511658 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1130/2240] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:36:14 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:36:14.517982 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ed475-d8f3-49bb-bc38-001fd866a0dc None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:14.529729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ed475-d8f3-49bb-bc38-001fd866a0dc None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:14.539206 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98253fb8-a534-4931-843a-a96d6de53a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:14.540436 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98253fb8-a534-4931-843a-a96d6de53a85 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:14.540912 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98253fb8-a534-4931-843a-a96d6de53a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:14.541320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98253fb8-a534-4931-843a-a96d6de53a85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:14.542119 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98253fb8-a534-4931-843a-a96d6de53a85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:14.544586 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98253fb8-a534-4931-843a-a96d6de53a85 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:14.611395 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1111/2241] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:36:14 2022] POST /identity/v3/projects => generated 374 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:36:14.612421 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f4c52-a612-4a7e-b48f-4522d016871b admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:14.626764 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f4c52-a612-4a7e-b48f-4522d016871b admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:14.702311 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f9516-4d31-4d06-99c6-be0315e754dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:14.703060 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f9516-4d31-4d06-99c6-be0315e754dc admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:14.703352 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f9516-4d31-4d06-99c6-be0315e754dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:14.703634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f9516-4d31-4d06-99c6-be0315e754dc admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:14.704523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f9516-4d31-4d06-99c6-be0315e754dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:14.706770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f9516-4d31-4d06-99c6-be0315e754dc admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:14.732052 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1131/2242] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:36:14 2022] POST /identity/v3/users => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:36:14.736953 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98253fb8-a534-4931-843a-a96d6de53a85 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:14.748415 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98253fb8-a534-4931-843a-a96d6de53a85 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:14.754129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f72dc-69cf-40ed-b28a-d99f83332682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:14.754926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f72dc-69cf-40ed-b28a-d99f83332682 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:14.755346 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f72dc-69cf-40ed-b28a-d99f83332682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:14.755644 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f72dc-69cf-40ed-b28a-d99f83332682 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:14.758840 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f72dc-69cf-40ed-b28a-d99f83332682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:14.758840 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f72dc-69cf-40ed-b28a-d99f83332682 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:14.775578 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1112/2243] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:36:14 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:36:14.777022 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f9516-4d31-4d06-99c6-be0315e754dc admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:14.788249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f9516-4d31-4d06-99c6-be0315e754dc admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:14.793297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc5e6a-c847-4aa3-bc37-c602d0ba0c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:14.793944 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc5e6a-c847-4aa3-bc37-c602d0ba0c9e admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:14.794355 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc5e6a-c847-4aa3-bc37-c602d0ba0c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:14.794637 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc5e6a-c847-4aa3-bc37-c602d0ba0c9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:14.795362 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc5e6a-c847-4aa3-bc37-c602d0ba0c9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:14.797053 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc5e6a-c847-4aa3-bc37-c602d0ba0c9e admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:14.809063 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1132/2244] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:36:14 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:36:14.813653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f72dc-69cf-40ed-b28a-d99f83332682 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:14.824503 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f72dc-69cf-40ed-b28a-d99f83332682 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:14.829594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdcf67f-b0fd-469d-9ff1-3b1bee5fa07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:14.830289 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdcf67f-b0fd-469d-9ff1-3b1bee5fa07b admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:14.830578 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdcf67f-b0fd-469d-9ff1-3b1bee5fa07b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:14.830867 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdcf67f-b0fd-469d-9ff1-3b1bee5fa07b admin admin] PATH_INFO: `/v3/projects/e0e158c30c2844bc88f8338bce9b6836/users/b2e6e27ed7b5486a9c28a944ea5c6004/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:14.847385 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdcf67f-b0fd-469d-9ff1-3b1bee5fa07b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0e158c30c2844bc88f8338bce9b6836, user_id=b2e6e27ed7b5486a9c28a944ea5c6004, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:14.850094 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdcf67f-b0fd-469d-9ff1-3b1bee5fa07b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:14.868767 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1113/2245] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:36:14 2022] PUT /identity/v3/projects/e0e158c30c2844bc88f8338bce9b6836/users/b2e6e27ed7b5486a9c28a944ea5c6004/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:14.881979 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c946b0b-74fb-449f-b42c-6e5a9d76d116 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:14.884083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c946b0b-74fb-449f-b42c-6e5a9d76d116 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:14.884083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c946b0b-74fb-449f-b42c-6e5a9d76d116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:14.919826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-0c946b0b-74fb-449f-b42c-6e5a9d76d116 None None] MFA Rules not processed for user `b2e6e27ed7b5486a9c28a944ea5c6004`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:14.921952 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-0c946b0b-74fb-449f-b42c-6e5a9d76d116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:14.952232 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:15.026068 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1133/2246] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:15.031210 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdcf67f-b0fd-469d-9ff1-3b1bee5fa07b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:15.102576 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdcf67f-b0fd-469d-9ff1-3b1bee5fa07b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:15.108195 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095fe62-3439-46be-87ea-02bb78558ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:15.108870 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095fe62-3439-46be-87ea-02bb78558ffd admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:15.109149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095fe62-3439-46be-87ea-02bb78558ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:15.109454 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095fe62-3439-46be-87ea-02bb78558ffd admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:15.110298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0095fe62-3439-46be-87ea-02bb78558ffd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:15.112781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0095fe62-3439-46be-87ea-02bb78558ffd admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:15.147827 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1114/2247] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:36:15 2022] POST /identity/v3/projects => generated 376 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:36:15.151622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c946b0b-74fb-449f-b42c-6e5a9d76d116 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:15.164028 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c946b0b-74fb-449f-b42c-6e5a9d76d116 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:15.241549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106cb5a-178e-4d62-9f47-d00c4c7bfb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:15.242206 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106cb5a-178e-4d62-9f47-d00c4c7bfb64 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:15.242494 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106cb5a-178e-4d62-9f47-d00c4c7bfb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:15.242773 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106cb5a-178e-4d62-9f47-d00c4c7bfb64 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:15.243497 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b106cb5a-178e-4d62-9f47-d00c4c7bfb64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:15.245527 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b106cb5a-178e-4d62-9f47-d00c4c7bfb64 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:15.274941 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1134/2248] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:36:15 2022] POST /identity/v3/users => generated 295 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:36:15.279689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095fe62-3439-46be-87ea-02bb78558ffd admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:15.291696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095fe62-3439-46be-87ea-02bb78558ffd admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:15.296979 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275a7f8-3556-488d-bd6f-f89dbdc91212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:15.297647 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275a7f8-3556-488d-bd6f-f89dbdc91212 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:15.298001 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275a7f8-3556-488d-bd6f-f89dbdc91212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:15.298282 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275a7f8-3556-488d-bd6f-f89dbdc91212 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:15.299191 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6275a7f8-3556-488d-bd6f-f89dbdc91212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:15.301033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6275a7f8-3556-488d-bd6f-f89dbdc91212 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:15.315599 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1115/2249] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:36:15 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:36:15.320872 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106cb5a-178e-4d62-9f47-d00c4c7bfb64 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:15.338220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106cb5a-178e-4d62-9f47-d00c4c7bfb64 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:15.352641 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e14e1f2-283f-46cd-91b4-efe350b47836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:15.353958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14e1f2-283f-46cd-91b4-efe350b47836 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:15.354544 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14e1f2-283f-46cd-91b4-efe350b47836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:15.355038 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14e1f2-283f-46cd-91b4-efe350b47836 admin admin] PATH_INFO: `/v3/projects/6260b65433eb4c408488fe48533098e1/users/e514c6485fa64306a3f7a560eb907c17/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:15.378533 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e14e1f2-283f-46cd-91b4-efe350b47836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6260b65433eb4c408488fe48533098e1, user_id=e514c6485fa64306a3f7a560eb907c17, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:15.382005 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e14e1f2-283f-46cd-91b4-efe350b47836 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:15.405716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1135/2250] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:36:15 2022] PUT /identity/v3/projects/6260b65433eb4c408488fe48533098e1/users/e514c6485fa64306a3f7a560eb907c17/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:15.410209 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275a7f8-3556-488d-bd6f-f89dbdc91212 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:15.421637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275a7f8-3556-488d-bd6f-f89dbdc91212 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:15.485536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:15.493294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266284a-d209-4a18-a808-65eb184f2c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:15.494147 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266284a-d209-4a18-a808-65eb184f2c64 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:15.494527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266284a-d209-4a18-a808-65eb184f2c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:15.495295 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266284a-d209-4a18-a808-65eb184f2c64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:15.496413 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a266284a-d209-4a18-a808-65eb184f2c64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:15.498049 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a266284a-d209-4a18-a808-65eb184f2c64 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:15.512288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1116/2251] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:36:15 2022] GET /identity/v3/roles => generated 1427 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:36:15.516738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e14e1f2-283f-46cd-91b4-efe350b47836 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:15.528249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e14e1f2-283f-46cd-91b4-efe350b47836 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:15.533604 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf8915-5dd7-4f11-9797-d08c3f4a4c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:15.534308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf8915-5dd7-4f11-9797-d08c3f4a4c95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:15.534623 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf8915-5dd7-4f11-9797-d08c3f4a4c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:15.534965 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf8915-5dd7-4f11-9797-d08c3f4a4c95 admin admin] PATH_INFO: `/v3/projects/6260b65433eb4c408488fe48533098e1/users/e514c6485fa64306a3f7a560eb907c17/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:15.537148 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abf8915-5dd7-4f11-9797-d08c3f4a4c95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6260b65433eb4c408488fe48533098e1, user_id=e514c6485fa64306a3f7a560eb907c17, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:15.540040 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abf8915-5dd7-4f11-9797-d08c3f4a4c95 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:15.558725 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1136/2252] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:36:15 2022] PUT /identity/v3/projects/6260b65433eb4c408488fe48533098e1/users/e514c6485fa64306a3f7a560eb907c17/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:15.570579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07fdf0-9f18-4ba9-b9e0-a157ace0544d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:15.570911 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07fdf0-9f18-4ba9-b9e0-a157ace0544d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:15.571138 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07fdf0-9f18-4ba9-b9e0-a157ace0544d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:15.606688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-bc07fdf0-9f18-4ba9-b9e0-a157ace0544d None None] MFA Rules not processed for user `e514c6485fa64306a3f7a560eb907c17`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:15.608006 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bc07fdf0-9f18-4ba9-b9e0-a157ace0544d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:15.699365 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1117/2253] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:15 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:15.713429 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf8915-5dd7-4f11-9797-d08c3f4a4c95 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:15.715628 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6abf8915-5dd7-4f11-9797-d08c3f4a4c95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:15.803389 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf8915-5dd7-4f11-9797-d08c3f4a4c95 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:15.809507 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a5b00-7f04-40fc-8ed9-21c36b0e98fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:15.810174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a5b00-7f04-40fc-8ed9-21c36b0e98fa service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:15.811633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a5b00-7f04-40fc-8ed9-21c36b0e98fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:15.812066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a5b00-7f04-40fc-8ed9-21c36b0e98fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:15.854147 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:15.893060 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a5b00-7f04-40fc-8ed9-21c36b0e98fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:15.895518 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a5b00-7f04-40fc-8ed9-21c36b0e98fa service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:15.906452 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1137/2254] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:36:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:36:16.588726 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07fdf0-9f18-4ba9-b9e0-a157ace0544d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:16.590769 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bc07fdf0-9f18-4ba9-b9e0-a157ace0544d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:16.681313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07fdf0-9f18-4ba9-b9e0-a157ace0544d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:16.686404 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d5646-ed38-4411-b840-99328afa4052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:16.687082 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d5646-ed38-4411-b840-99328afa4052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:16.687360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d5646-ed38-4411-b840-99328afa4052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:16.687635 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d5646-ed38-4411-b840-99328afa4052 admin admin] PATH_INFO: `/v3/users/73c7ca3c28a14b7e8e59d9b3a1d806b9` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:16.688938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d5646-ed38-4411-b840-99328afa4052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73c7ca3c28a14b7e8e59d9b3a1d806b9)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:16.691168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d5646-ed38-4411-b840-99328afa4052 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:16.759787 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6d7d5646-ed38-4411-b840-99328afa4052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73c7ca3c28a14b7e8e59d9b3a1d806b9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:16.769202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6d7d5646-ed38-4411-b840-99328afa4052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73c7ca3c28a14b7e8e59d9b3a1d806b9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:16.770053 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6d7d5646-ed38-4411-b840-99328afa4052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73c7ca3c28a14b7e8e59d9b3a1d806b9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:16.782445 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6d7d5646-ed38-4411-b840-99328afa4052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73c7ca3c28a14b7e8e59d9b3a1d806b9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:16.783194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6d7d5646-ed38-4411-b840-99328afa4052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73c7ca3c28a14b7e8e59d9b3a1d806b9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:16.793348 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:16.800420 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1118/2255] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:36:16 2022] DELETE /identity/v3/users/73c7ca3c28a14b7e8e59d9b3a1d806b9 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:16.805333 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9df000-fd41-49f2-b1eb-245bae6dcdee None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:16.805920 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9df000-fd41-49f2-b1eb-245bae6dcdee None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:16.805920 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9df000-fd41-49f2-b1eb-245bae6dcdee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:16.826135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-fc9df000-fd41-49f2-b1eb-245bae6dcdee None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:16.827411 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-fc9df000-fd41-49f2-b1eb-245bae6dcdee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:16.944250 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1138/2256] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:16.962477 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d5646-ed38-4411-b840-99328afa4052 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:16.964606 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6d7d5646-ed38-4411-b840-99328afa4052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:17.067180 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d5646-ed38-4411-b840-99328afa4052 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:17.080768 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95558ed-f2bd-4f11-852a-39c02ff1a3ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:17.080768 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95558ed-f2bd-4f11-852a-39c02ff1a3ea service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:17.080768 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95558ed-f2bd-4f11-852a-39c02ff1a3ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:17.080768 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95558ed-f2bd-4f11-852a-39c02ff1a3ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:17.089896 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95558ed-f2bd-4f11-852a-39c02ff1a3ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:17.093021 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95558ed-f2bd-4f11-852a-39c02ff1a3ea service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:17.114486 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1119/2257] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:36:16 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:36:17.563638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9df000-fd41-49f2-b1eb-245bae6dcdee None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:17.565510 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-fc9df000-fd41-49f2-b1eb-245bae6dcdee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:17.579816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9df000-fd41-49f2-b1eb-245bae6dcdee None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:17.595901 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ee128-f1c6-4692-9cbc-819621cb31f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:17.597214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ee128-f1c6-4692-9cbc-819621cb31f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:17.597592 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ee128-f1c6-4692-9cbc-819621cb31f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:17.598063 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ee128-f1c6-4692-9cbc-819621cb31f3 admin admin] PATH_INFO: `/v3/projects/b5d35bf9fba545168762f4714ed2cac2` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:17.609354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8ee128-f1c6-4692-9cbc-819621cb31f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5d35bf9fba545168762f4714ed2cac2)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:17.613310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8ee128-f1c6-4692-9cbc-819621cb31f3 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:17.671247 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7a8ee128-f1c6-4692-9cbc-819621cb31f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5d35bf9fba545168762f4714ed2cac2', 'request_id': 'req-7a8ee128-f1c6-4692-9cbc-819621cb31f3'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:17.680301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7a8ee128-f1c6-4692-9cbc-819621cb31f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5d35bf9fba545168762f4714ed2cac2', 'request_id': 'req-7a8ee128-f1c6-4692-9cbc-819621cb31f3'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:17.687408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7a8ee128-f1c6-4692-9cbc-819621cb31f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5d35bf9fba545168762f4714ed2cac2', 'request_id': 'req-7a8ee128-f1c6-4692-9cbc-819621cb31f3'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:17.701328 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7a8ee128-f1c6-4692-9cbc-819621cb31f3 admin admin] The token cache is being invalidate because project b5d35bf9fba545168762f4714ed2cac2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:36:17.701647 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7a8ee128-f1c6-4692-9cbc-819621cb31f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:17.702295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7a8ee128-f1c6-4692-9cbc-819621cb31f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:17.703906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1139/2258] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:36:17 2022] DELETE /identity/v3/projects/b5d35bf9fba545168762f4714ed2cac2 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:36:18.604043 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95558ed-f2bd-4f11-852a-39c02ff1a3ea service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:18.606158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a95558ed-f2bd-4f11-852a-39c02ff1a3ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:18.626214 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95558ed-f2bd-4f11-852a-39c02ff1a3ea service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:18.735285 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a0e6e-d8af-4aa3-8e90-03cb5a14085f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:18.736472 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a0e6e-d8af-4aa3-8e90-03cb5a14085f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:18.736949 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a0e6e-d8af-4aa3-8e90-03cb5a14085f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:18.737499 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a0e6e-d8af-4aa3-8e90-03cb5a14085f admin admin] PATH_INFO: `/v3/users/389ed2f475a3479e821e777b1c4cafb0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:18.739842 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a0e6e-d8af-4aa3-8e90-03cb5a14085f admin admin] RBAC: Authorizing `identity:delete_user(user_id=389ed2f475a3479e821e777b1c4cafb0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:18.750426 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a0e6e-d8af-4aa3-8e90-03cb5a14085f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:18.822589 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3f4a0e6e-d8af-4aa3-8e90-03cb5a14085f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '389ed2f475a3479e821e777b1c4cafb0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:18.831333 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3f4a0e6e-d8af-4aa3-8e90-03cb5a14085f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '389ed2f475a3479e821e777b1c4cafb0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:18.832280 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3f4a0e6e-d8af-4aa3-8e90-03cb5a14085f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '389ed2f475a3479e821e777b1c4cafb0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:18.846606 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3f4a0e6e-d8af-4aa3-8e90-03cb5a14085f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '389ed2f475a3479e821e777b1c4cafb0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:18.847479 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3f4a0e6e-d8af-4aa3-8e90-03cb5a14085f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '389ed2f475a3479e821e777b1c4cafb0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:18.867098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1120/2259] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:36:18 2022] DELETE /identity/v3/users/389ed2f475a3479e821e777b1c4cafb0 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:18.873466 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae532f-4ca1-4f85-96a2-27f132471e54 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:18.874051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae532f-4ca1-4f85-96a2-27f132471e54 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:18.874152 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae532f-4ca1-4f85-96a2-27f132471e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:18.903274 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-11ae532f-4ca1-4f85-96a2-27f132471e54 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:18.904821 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-11ae532f-4ca1-4f85-96a2-27f132471e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:18.969720 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:18.989103 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1140/2260] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:18 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:19.003764 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a0e6e-d8af-4aa3-8e90-03cb5a14085f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:19.004570 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3f4a0e6e-d8af-4aa3-8e90-03cb5a14085f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:19.035656 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:19.081413 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a0e6e-d8af-4aa3-8e90-03cb5a14085f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:19.086577 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe44c9-baff-46ca-b54f-dc2c2258a2e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:19.087281 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe44c9-baff-46ca-b54f-dc2c2258a2e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:19.087642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe44c9-baff-46ca-b54f-dc2c2258a2e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:19.088016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe44c9-baff-46ca-b54f-dc2c2258a2e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:19.101925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effe44c9-baff-46ca-b54f-dc2c2258a2e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:19.103830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effe44c9-baff-46ca-b54f-dc2c2258a2e3 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:19.118492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1121/2261] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:36:19 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:36:19.441842 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae532f-4ca1-4f85-96a2-27f132471e54 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:19.444037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-11ae532f-4ca1-4f85-96a2-27f132471e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:19.458461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae532f-4ca1-4f85-96a2-27f132471e54 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:19.465782 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb410c2-b286-4001-9077-c09301e6efd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:19.466492 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb410c2-b286-4001-9077-c09301e6efd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:19.466787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb410c2-b286-4001-9077-c09301e6efd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:19.467098 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb410c2-b286-4001-9077-c09301e6efd0 admin admin] PATH_INFO: `/v3/projects/c8e2c1ea53cc4163a7df3474d1498149` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:19.468695 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb410c2-b286-4001-9077-c09301e6efd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8e2c1ea53cc4163a7df3474d1498149)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:19.470731 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb410c2-b286-4001-9077-c09301e6efd0 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:19.536266 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-9bb410c2-b286-4001-9077-c09301e6efd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8e2c1ea53cc4163a7df3474d1498149', 'request_id': 'req-9bb410c2-b286-4001-9077-c09301e6efd0'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:19.546112 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-9bb410c2-b286-4001-9077-c09301e6efd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8e2c1ea53cc4163a7df3474d1498149', 'request_id': 'req-9bb410c2-b286-4001-9077-c09301e6efd0'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:19.553353 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-9bb410c2-b286-4001-9077-c09301e6efd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8e2c1ea53cc4163a7df3474d1498149', 'request_id': 'req-9bb410c2-b286-4001-9077-c09301e6efd0'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:19.566286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-9bb410c2-b286-4001-9077-c09301e6efd0 admin admin] The token cache is being invalidate because project c8e2c1ea53cc4163a7df3474d1498149 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:36:19.566571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-9bb410c2-b286-4001-9077-c09301e6efd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:19.567261 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-9bb410c2-b286-4001-9077-c09301e6efd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:19.568908 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1141/2262] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:36:19 2022] DELETE /identity/v3/projects/c8e2c1ea53cc4163a7df3474d1498149 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:19.575621 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c3c5b-191c-4e80-97c5-91cc5c955025 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:19.575902 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c3c5b-191c-4e80-97c5-91cc5c955025 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:19.576232 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c3c5b-191c-4e80-97c5-91cc5c955025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:19.598710 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c65c3c5b-191c-4e80-97c5-91cc5c955025 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:19.600539 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c65c3c5b-191c-4e80-97c5-91cc5c955025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:19.691024 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1122/2263] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:19.710649 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428dbc0-74c3-4911-ab84-f9766d34a24c None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:19.710946 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428dbc0-74c3-4911-ab84-f9766d34a24c None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:19.711232 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428dbc0-74c3-4911-ab84-f9766d34a24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:19.740455 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-d428dbc0-74c3-4911-ab84-f9766d34a24c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:19.741742 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d428dbc0-74c3-4911-ab84-f9766d34a24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:19.751323 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1142/2264] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:19.756404 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428dbc0-74c3-4911-ab84-f9766d34a24c None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:19.756579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c3c5b-191c-4e80-97c5-91cc5c955025 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:19.758123 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c65c3c5b-191c-4e80-97c5-91cc5c955025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:19.766425 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428dbc0-74c3-4911-ab84-f9766d34a24c None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:19.770947 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c3c5b-191c-4e80-97c5-91cc5c955025 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:19.773257 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f52e8-4cda-4286-bad6-0711194e8ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:19.774220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f52e8-4cda-4286-bad6-0711194e8ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:19.774570 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f52e8-4cda-4286-bad6-0711194e8ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:19.774919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f52e8-4cda-4286-bad6-0711194e8ee8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:19.775957 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f52e8-4cda-4286-bad6-0711194e8ee8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:36:19.776374 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f52e8-4cda-4286-bad6-0711194e8ee8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:19.777715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36c419-0314-4e16-b372-5aac3fc1575c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:19.778357 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f52e8-4cda-4286-bad6-0711194e8ee8 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:19.778489 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36c419-0314-4e16-b372-5aac3fc1575c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:19.778654 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36c419-0314-4e16-b372-5aac3fc1575c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:19.778946 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36c419-0314-4e16-b372-5aac3fc1575c admin admin] PATH_INFO: `/v3/users/b2e6e27ed7b5486a9c28a944ea5c6004` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:19.780326 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36c419-0314-4e16-b372-5aac3fc1575c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2e6e27ed7b5486a9c28a944ea5c6004)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:19.782316 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36c419-0314-4e16-b372-5aac3fc1575c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:19.793409 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1143/2265] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:36:19 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:36:19.798036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f52e8-4cda-4286-bad6-0711194e8ee8 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:19.810656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f52e8-4cda-4286-bad6-0711194e8ee8 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:19.820670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c54921-3a31-4db0-a37e-5eb4857a0ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:19.821626 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c54921-3a31-4db0-a37e-5eb4857a0ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:19.821811 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c54921-3a31-4db0-a37e-5eb4857a0ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:19.822138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c54921-3a31-4db0-a37e-5eb4857a0ca6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:19.822941 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c54921-3a31-4db0-a37e-5eb4857a0ca6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:19.825162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c54921-3a31-4db0-a37e-5eb4857a0ca6 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:19.849418 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ab36c419-0314-4e16-b372-5aac3fc1575c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2e6e27ed7b5486a9c28a944ea5c6004'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:19.859469 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ab36c419-0314-4e16-b372-5aac3fc1575c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2e6e27ed7b5486a9c28a944ea5c6004'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:19.860400 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ab36c419-0314-4e16-b372-5aac3fc1575c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2e6e27ed7b5486a9c28a944ea5c6004'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:19.867530 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1144/2266] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:36:19 2022] POST /identity/v3/projects => generated 376 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:36:19.868718 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c54921-3a31-4db0-a37e-5eb4857a0ca6 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:19.870508 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-98c54921-3a31-4db0-a37e-5eb4857a0ca6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:19.877035 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ab36c419-0314-4e16-b372-5aac3fc1575c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2e6e27ed7b5486a9c28a944ea5c6004'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:19.879076 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ab36c419-0314-4e16-b372-5aac3fc1575c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2e6e27ed7b5486a9c28a944ea5c6004'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:19.898810 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1123/2267] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:36:19 2022] DELETE /identity/v3/users/b2e6e27ed7b5486a9c28a944ea5c6004 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:19.904829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40e2f2-ee94-4989-8d89-9b77636b9059 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:19.905266 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40e2f2-ee94-4989-8d89-9b77636b9059 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:19.905658 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40e2f2-ee94-4989-8d89-9b77636b9059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:19.932217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ef40e2f2-ee94-4989-8d89-9b77636b9059 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:19.934019 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ef40e2f2-ee94-4989-8d89-9b77636b9059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:19.972360 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c54921-3a31-4db0-a37e-5eb4857a0ca6 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:19.972908 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:20.024709 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:20.030868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1124/2268] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:20.044589 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40e2f2-ee94-4989-8d89-9b77636b9059 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:20.046559 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ef40e2f2-ee94-4989-8d89-9b77636b9059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:20.058900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d738f7b-a3da-4229-b413-734946955e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:20.059824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d738f7b-a3da-4229-b413-734946955e1f admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:20.060069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d738f7b-a3da-4229-b413-734946955e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:20.060453 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d738f7b-a3da-4229-b413-734946955e1f admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:20.061201 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d738f7b-a3da-4229-b413-734946955e1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:20.063305 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d738f7b-a3da-4229-b413-734946955e1f admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:20.100041 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1145/2269] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:36:19 2022] POST /identity/v3/users => generated 289 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:36:20.102522 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d738f7b-a3da-4229-b413-734946955e1f admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:20.116020 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d738f7b-a3da-4229-b413-734946955e1f admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:20.120849 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9f9f1-4d8c-478e-8ccc-3bc1a014e466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:20.121061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40e2f2-ee94-4989-8d89-9b77636b9059 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:20.121490 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9f9f1-4d8c-478e-8ccc-3bc1a014e466 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:20.121765 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9f9f1-4d8c-478e-8ccc-3bc1a014e466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:20.122037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9f9f1-4d8c-478e-8ccc-3bc1a014e466 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:20.122756 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9f9f1-4d8c-478e-8ccc-3bc1a014e466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:20.124421 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9f9f1-4d8c-478e-8ccc-3bc1a014e466 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:20.124745 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b3e8b-b6cb-4cc1-aceb-dcf81703a2e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:20.125382 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b3e8b-b6cb-4cc1-aceb-dcf81703a2e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:20.125666 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b3e8b-b6cb-4cc1-aceb-dcf81703a2e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:20.125950 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b3e8b-b6cb-4cc1-aceb-dcf81703a2e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:20.136055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1146/2270] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:36:20 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:36:20.139580 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b3e8b-b6cb-4cc1-aceb-dcf81703a2e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:20.139580 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b3e8b-b6cb-4cc1-aceb-dcf81703a2e0 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:20.140482 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9f9f1-4d8c-478e-8ccc-3bc1a014e466 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:20.149050 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1125/2271] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:36:20 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:36:20.151036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9f9f1-4d8c-478e-8ccc-3bc1a014e466 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:20.155729 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec8fa7-5c25-420d-ab20-0dd6f8dfac55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:20.156546 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec8fa7-5c25-420d-ab20-0dd6f8dfac55 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:20.156976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec8fa7-5c25-420d-ab20-0dd6f8dfac55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:20.157296 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec8fa7-5c25-420d-ab20-0dd6f8dfac55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:20.158008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec8fa7-5c25-420d-ab20-0dd6f8dfac55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:20.160002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec8fa7-5c25-420d-ab20-0dd6f8dfac55 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:20.171770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1147/2272] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:36:20 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:36:20.176930 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b3e8b-b6cb-4cc1-aceb-dcf81703a2e0 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:20.187748 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b3e8b-b6cb-4cc1-aceb-dcf81703a2e0 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:20.197906 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf6cef-ef5b-4db0-9179-8b013eed8b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:20.198865 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf6cef-ef5b-4db0-9179-8b013eed8b9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:20.199260 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf6cef-ef5b-4db0-9179-8b013eed8b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:20.199672 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf6cef-ef5b-4db0-9179-8b013eed8b9b admin admin] PATH_INFO: `/v3/projects/3c41c04883804e9586e241166703c1ce/users/c06eaea526a346b3b86f27eb347b9856/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:20.223204 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf6cef-ef5b-4db0-9179-8b013eed8b9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c41c04883804e9586e241166703c1ce, user_id=c06eaea526a346b3b86f27eb347b9856, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:20.227543 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf6cef-ef5b-4db0-9179-8b013eed8b9b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:20.254524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1126/2273] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:36:20 2022] PUT /identity/v3/projects/3c41c04883804e9586e241166703c1ce/users/c06eaea526a346b3b86f27eb347b9856/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:36:20.266832 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc193f0-124b-4c88-9a08-47bba76c30cb None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:20.267367 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc193f0-124b-4c88-9a08-47bba76c30cb None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:20.267682 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc193f0-124b-4c88-9a08-47bba76c30cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:20.305095 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-bfc193f0-124b-4c88-9a08-47bba76c30cb None None] MFA Rules not processed for user `c06eaea526a346b3b86f27eb347b9856`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:20.307239 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-bfc193f0-124b-4c88-9a08-47bba76c30cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:20.396377 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1148/2274] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:36:20.408990 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf6cef-ef5b-4db0-9179-8b013eed8b9b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:20.413069 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-94bf6cef-ef5b-4db0-9179-8b013eed8b9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:20.508812 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf6cef-ef5b-4db0-9179-8b013eed8b9b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:20.515100 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0921ed5-7079-4a1f-9368-3ee1eda9aef4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:20.515868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0921ed5-7079-4a1f-9368-3ee1eda9aef4 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:20.516253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0921ed5-7079-4a1f-9368-3ee1eda9aef4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:20.516694 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0921ed5-7079-4a1f-9368-3ee1eda9aef4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:20.527324 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0921ed5-7079-4a1f-9368-3ee1eda9aef4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:20.528853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0921ed5-7079-4a1f-9368-3ee1eda9aef4 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:20.541052 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1127/2275] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:36:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:36:20.554110 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc193f0-124b-4c88-9a08-47bba76c30cb None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:20.566321 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-bfc193f0-124b-4c88-9a08-47bba76c30cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:20.676541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc193f0-124b-4c88-9a08-47bba76c30cb None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:20.681716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e46a00-386e-443a-9a37-f31edffa662f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:20.682577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e46a00-386e-443a-9a37-f31edffa662f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:20.682869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e46a00-386e-443a-9a37-f31edffa662f admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:20.683153 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e46a00-386e-443a-9a37-f31edffa662f admin admin] PATH_INFO: `/v3/projects/e0e158c30c2844bc88f8338bce9b6836` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:20.684479 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e46a00-386e-443a-9a37-f31edffa662f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0e158c30c2844bc88f8338bce9b6836)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:20.686545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e46a00-386e-443a-9a37-f31edffa662f admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:20.764893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:20.785277 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e8e46a00-386e-443a-9a37-f31edffa662f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0e158c30c2844bc88f8338bce9b6836', 'request_id': 'req-e8e46a00-386e-443a-9a37-f31edffa662f'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:20.797049 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e8e46a00-386e-443a-9a37-f31edffa662f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0e158c30c2844bc88f8338bce9b6836', 'request_id': 'req-e8e46a00-386e-443a-9a37-f31edffa662f'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:20.807353 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e8e46a00-386e-443a-9a37-f31edffa662f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0e158c30c2844bc88f8338bce9b6836', 'request_id': 'req-e8e46a00-386e-443a-9a37-f31edffa662f'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:20.828413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e8e46a00-386e-443a-9a37-f31edffa662f admin admin] The token cache is being invalidate because project e0e158c30c2844bc88f8338bce9b6836 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:36:20.828770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e8e46a00-386e-443a-9a37-f31edffa662f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:20.829570 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e8e46a00-386e-443a-9a37-f31edffa662f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:20.832813 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1149/2276] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:36:20 2022] DELETE /identity/v3/projects/e0e158c30c2844bc88f8338bce9b6836 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:20.842790 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0921ed5-7079-4a1f-9368-3ee1eda9aef4 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:20.859785 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0921ed5-7079-4a1f-9368-3ee1eda9aef4 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:20.943801 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:20.953375 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2819d-5844-4fc9-856f-f08331f8dbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:20.954540 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2819d-5844-4fc9-856f-f08331f8dbc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:20.954933 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2819d-5844-4fc9-856f-f08331f8dbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:20.955278 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2819d-5844-4fc9-856f-f08331f8dbc5 admin admin] PATH_INFO: `/v3/users/e514c6485fa64306a3f7a560eb907c17` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:20.956849 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb2819d-5844-4fc9-856f-f08331f8dbc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e514c6485fa64306a3f7a560eb907c17)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:20.958974 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb2819d-5844-4fc9-856f-f08331f8dbc5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:21.041171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fcb2819d-5844-4fc9-856f-f08331f8dbc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e514c6485fa64306a3f7a560eb907c17'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:21.052272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fcb2819d-5844-4fc9-856f-f08331f8dbc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e514c6485fa64306a3f7a560eb907c17'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:21.053277 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fcb2819d-5844-4fc9-856f-f08331f8dbc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e514c6485fa64306a3f7a560eb907c17'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:21.093958 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fcb2819d-5844-4fc9-856f-f08331f8dbc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e514c6485fa64306a3f7a560eb907c17'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:21.094758 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fcb2819d-5844-4fc9-856f-f08331f8dbc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e514c6485fa64306a3f7a560eb907c17'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:21.130879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1128/2277] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:36:20 2022] DELETE /identity/v3/users/e514c6485fa64306a3f7a560eb907c17 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:36:21.486764 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e46a00-386e-443a-9a37-f31edffa662f admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:21.488348 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-e8e46a00-386e-443a-9a37-f31edffa662f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:21.505388 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e46a00-386e-443a-9a37-f31edffa662f admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:21.581753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136cb0bd-fd8c-4516-aa18-216a7f3549c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:21.582498 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cb0bd-fd8c-4516-aa18-216a7f3549c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:21.582719 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cb0bd-fd8c-4516-aa18-216a7f3549c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:21.583003 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cb0bd-fd8c-4516-aa18-216a7f3549c5 admin admin] PATH_INFO: `/v3/projects/6260b65433eb4c408488fe48533098e1` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:21.584331 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136cb0bd-fd8c-4516-aa18-216a7f3549c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6260b65433eb4c408488fe48533098e1)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:21.586460 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136cb0bd-fd8c-4516-aa18-216a7f3549c5 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:21.642853 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-136cb0bd-fd8c-4516-aa18-216a7f3549c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6260b65433eb4c408488fe48533098e1', 'request_id': 'req-136cb0bd-fd8c-4516-aa18-216a7f3549c5'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:21.651706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-136cb0bd-fd8c-4516-aa18-216a7f3549c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6260b65433eb4c408488fe48533098e1', 'request_id': 'req-136cb0bd-fd8c-4516-aa18-216a7f3549c5'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:21.658220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-136cb0bd-fd8c-4516-aa18-216a7f3549c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6260b65433eb4c408488fe48533098e1', 'request_id': 'req-136cb0bd-fd8c-4516-aa18-216a7f3549c5'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:21.670703 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-136cb0bd-fd8c-4516-aa18-216a7f3549c5 admin admin] The token cache is being invalidate because project 6260b65433eb4c408488fe48533098e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:36:21.670991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-136cb0bd-fd8c-4516-aa18-216a7f3549c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:21.671699 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-136cb0bd-fd8c-4516-aa18-216a7f3549c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:21.673340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1150/2278] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:36:21 2022] DELETE /identity/v3/projects/6260b65433eb4c408488fe48533098e1 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:21.679061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fa79b-e33a-48d7-a5ac-d89fe4b3e54b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:21.679355 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fa79b-e33a-48d7-a5ac-d89fe4b3e54b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:21.679636 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fa79b-e33a-48d7-a5ac-d89fe4b3e54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:21.704388 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6c9fa79b-e33a-48d7-a5ac-d89fe4b3e54b None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:21.705659 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6c9fa79b-e33a-48d7-a5ac-d89fe4b3e54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:21.771122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1129/2279] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:21 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:21.784261 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea1e9d-d8dd-4251-a304-9dbe030a1d76 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:21.784605 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea1e9d-d8dd-4251-a304-9dbe030a1d76 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:21.784886 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea1e9d-d8dd-4251-a304-9dbe030a1d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:21.807115 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-e6ea1e9d-d8dd-4251-a304-9dbe030a1d76 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:21.808433 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-e6ea1e9d-d8dd-4251-a304-9dbe030a1d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:21.817388 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1151/2280] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:21 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:21.823655 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fa79b-e33a-48d7-a5ac-d89fe4b3e54b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:21.837010 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fa79b-e33a-48d7-a5ac-d89fe4b3e54b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:21.841970 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494be5ad-a0b8-4dae-a810-6b392db99dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:21.842617 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494be5ad-a0b8-4dae-a810-6b392db99dfa admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:21.842876 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494be5ad-a0b8-4dae-a810-6b392db99dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:21.843530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494be5ad-a0b8-4dae-a810-6b392db99dfa admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:21.843930 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494be5ad-a0b8-4dae-a810-6b392db99dfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:36:21.844290 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494be5ad-a0b8-4dae-a810-6b392db99dfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:21.845949 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494be5ad-a0b8-4dae-a810-6b392db99dfa admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:21.860134 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1130/2281] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:36:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:36:21.865430 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea1e9d-d8dd-4251-a304-9dbe030a1d76 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:21.876363 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea1e9d-d8dd-4251-a304-9dbe030a1d76 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:21.881562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd756e-c60f-4e31-adea-d4033af99f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:21.882332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd756e-c60f-4e31-adea-d4033af99f21 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:21.882640 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd756e-c60f-4e31-adea-d4033af99f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:21.882986 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd756e-c60f-4e31-adea-d4033af99f21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:21.883761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd756e-c60f-4e31-adea-d4033af99f21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:21.885936 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd756e-c60f-4e31-adea-d4033af99f21 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:21.929691 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1152/2282] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:36:21 2022] POST /identity/v3/projects => generated 384 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:36:21.934216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494be5ad-a0b8-4dae-a810-6b392db99dfa admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:21.946419 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494be5ad-a0b8-4dae-a810-6b392db99dfa admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:22.019369 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea88246-06b3-43cb-9220-42fbc67ebd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:22.020067 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea88246-06b3-43cb-9220-42fbc67ebd86 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:22.020361 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea88246-06b3-43cb-9220-42fbc67ebd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:22.020642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea88246-06b3-43cb-9220-42fbc67ebd86 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:22.021350 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea88246-06b3-43cb-9220-42fbc67ebd86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:22.023307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea88246-06b3-43cb-9220-42fbc67ebd86 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:22.035182 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:22.057402 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1131/2283] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:36:21 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:36:22.059326 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd756e-c60f-4e31-adea-d4033af99f21 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:22.078212 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd756e-c60f-4e31-adea-d4033af99f21 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:22.083529 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d65724-c8e9-4efc-80bc-51922a001ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:22.084440 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d65724-c8e9-4efc-80bc-51922a001ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:22.084572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d65724-c8e9-4efc-80bc-51922a001ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:22.084760 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d65724-c8e9-4efc-80bc-51922a001ad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:22.085478 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d65724-c8e9-4efc-80bc-51922a001ad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:22.087202 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d65724-c8e9-4efc-80bc-51922a001ad0 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:22.103900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1153/2284] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:36:22 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:36:22.105102 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea88246-06b3-43cb-9220-42fbc67ebd86 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:22.122204 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea88246-06b3-43cb-9220-42fbc67ebd86 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:22.127504 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43378e28-ccd6-4c9d-9f95-abc1b767f0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:22.128252 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43378e28-ccd6-4c9d-9f95-abc1b767f0a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:22.128578 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43378e28-ccd6-4c9d-9f95-abc1b767f0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:22.128840 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43378e28-ccd6-4c9d-9f95-abc1b767f0a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:22.129561 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43378e28-ccd6-4c9d-9f95-abc1b767f0a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:22.131196 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43378e28-ccd6-4c9d-9f95-abc1b767f0a1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:22.143195 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1132/2285] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:36:22 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:36:22.147993 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d65724-c8e9-4efc-80bc-51922a001ad0 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:22.160186 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d65724-c8e9-4efc-80bc-51922a001ad0 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:22.165840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec47b54-07d4-4f5f-8129-0736561cd2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:22.166565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec47b54-07d4-4f5f-8129-0736561cd2a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:22.166862 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec47b54-07d4-4f5f-8129-0736561cd2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:22.167129 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec47b54-07d4-4f5f-8129-0736561cd2a2 admin admin] PATH_INFO: `/v3/projects/8e9d4ca15763492bad22165413bae2bd/users/019980a5aef94a7ba95722964b472679/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:22.185813 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec47b54-07d4-4f5f-8129-0736561cd2a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e9d4ca15763492bad22165413bae2bd, user_id=019980a5aef94a7ba95722964b472679, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:22.189313 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec47b54-07d4-4f5f-8129-0736561cd2a2 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:22.193068 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:22.212041 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1154/2286] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:36:22 2022] PUT /identity/v3/projects/8e9d4ca15763492bad22165413bae2bd/users/019980a5aef94a7ba95722964b472679/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:22.233766 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322c9e6-bde9-4204-b1b0-2d29a70119f4 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:22.234072 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322c9e6-bde9-4204-b1b0-2d29a70119f4 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:22.234359 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322c9e6-bde9-4204-b1b0-2d29a70119f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:22.274689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7322c9e6-bde9-4204-b1b0-2d29a70119f4 None None] MFA Rules not processed for user `019980a5aef94a7ba95722964b472679`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:22.275971 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7322c9e6-bde9-4204-b1b0-2d29a70119f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:22.393577 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1133/2287] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:22.412618 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec47b54-07d4-4f5f-8129-0736561cd2a2 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:22.415266 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-3ec47b54-07d4-4f5f-8129-0736561cd2a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:22.495291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec47b54-07d4-4f5f-8129-0736561cd2a2 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:22.499666 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e04b3c-8d77-4f2b-9f12-cdf588e0001a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:22.500367 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e04b3c-8d77-4f2b-9f12-cdf588e0001a service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:22.500680 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e04b3c-8d77-4f2b-9f12-cdf588e0001a service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:22.500992 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e04b3c-8d77-4f2b-9f12-cdf588e0001a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:22.511165 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e04b3c-8d77-4f2b-9f12-cdf588e0001a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:22.512731 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e04b3c-8d77-4f2b-9f12-cdf588e0001a service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:22.523315 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1155/2288] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:36:22 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:36:23.180222 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1e9f9-2206-4713-a17b-d9b599b17ef6 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:23.180848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1e9f9-2206-4713-a17b-d9b599b17ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:23.180848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1e9f9-2206-4713-a17b-d9b599b17ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:23.205810 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-92e1e9f9-2206-4713-a17b-d9b599b17ef6 None None] MFA Rules not processed for user `c06eaea526a346b3b86f27eb347b9856`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:23.207393 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-92e1e9f9-2206-4713-a17b-d9b599b17ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:23.295716 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1134/2289] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:26.425047 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e04b3c-8d77-4f2b-9f12-cdf588e0001a service cinder] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:26.569449 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e04b3c-8d77-4f2b-9f12-cdf588e0001a service cinder] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:26.585009 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37da5c7-fe61-45ac-91ec-8b0bc1b4ef6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:26.586570 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37da5c7-fe61-45ac-91ec-8b0bc1b4ef6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:26.586853 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37da5c7-fe61-45ac-91ec-8b0bc1b4ef6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:26.587601 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37da5c7-fe61-45ac-91ec-8b0bc1b4ef6c admin admin] PATH_INFO: `/v3/users/019980a5aef94a7ba95722964b472679` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:26.592123 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37da5c7-fe61-45ac-91ec-8b0bc1b4ef6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=019980a5aef94a7ba95722964b472679)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:26.597852 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37da5c7-fe61-45ac-91ec-8b0bc1b4ef6c admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:26.709591 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a37da5c7-fe61-45ac-91ec-8b0bc1b4ef6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '019980a5aef94a7ba95722964b472679'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:26.710542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a37da5c7-fe61-45ac-91ec-8b0bc1b4ef6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '019980a5aef94a7ba95722964b472679'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:26.732903 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a37da5c7-fe61-45ac-91ec-8b0bc1b4ef6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '019980a5aef94a7ba95722964b472679'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:26.745759 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a37da5c7-fe61-45ac-91ec-8b0bc1b4ef6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '019980a5aef94a7ba95722964b472679'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:26.762196 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a37da5c7-fe61-45ac-91ec-8b0bc1b4ef6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '019980a5aef94a7ba95722964b472679'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:26.771880 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1156/2290] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:36:26 2022] DELETE /identity/v3/users/019980a5aef94a7ba95722964b472679 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:26.778907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd99cf-6a4c-49f1-8ca0-e6a039b01075 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:26.780109 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd99cf-6a4c-49f1-8ca0-e6a039b01075 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:26.780798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd99cf-6a4c-49f1-8ca0-e6a039b01075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:26.804315 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-67dd99cf-6a4c-49f1-8ca0-e6a039b01075 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:26.806085 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-67dd99cf-6a4c-49f1-8ca0-e6a039b01075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:26.894324 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1135/2291] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:26 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:26.912031 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37da5c7-fe61-45ac-91ec-8b0bc1b4ef6c admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:26.913955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-a37da5c7-fe61-45ac-91ec-8b0bc1b4ef6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:26.938605 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:27.022794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37da5c7-fe61-45ac-91ec-8b0bc1b4ef6c admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:27.027849 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6af95-c5d3-4e35-a62d-41596e847ab3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:27.028425 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6af95-c5d3-4e35-a62d-41596e847ab3 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:27.028715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6af95-c5d3-4e35-a62d-41596e847ab3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:27.029084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6af95-c5d3-4e35-a62d-41596e847ab3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:27.040756 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6af95-c5d3-4e35-a62d-41596e847ab3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:27.043838 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6af95-c5d3-4e35-a62d-41596e847ab3 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:27.057439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1157/2292] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:36:26 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:36:27.118688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a75035-e9c4-452a-901d-8161929a551d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:27.119315 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a75035-e9c4-452a-901d-8161929a551d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:27.121251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a75035-e9c4-452a-901d-8161929a551d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:27.149976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-98a75035-e9c4-452a-901d-8161929a551d None None] MFA Rules not processed for user `3170dc3323f54512bc8819c8efa86b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:27.151260 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-98a75035-e9c4-452a-901d-8161929a551d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:27.173874 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:27.267175 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1136/2293] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:27 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:27.704343 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6af95-c5d3-4e35-a62d-41596e847ab3 service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:27.707104 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-a3a6af95-c5d3-4e35-a62d-41596e847ab3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:27.731213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6af95-c5d3-4e35-a62d-41596e847ab3 service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:27.752216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604ac45-2b80-4b02-a795-27a8bb08e7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:27.753041 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604ac45-2b80-4b02-a795-27a8bb08e7d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:27.753413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604ac45-2b80-4b02-a795-27a8bb08e7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:27.753811 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604ac45-2b80-4b02-a795-27a8bb08e7d7 admin admin] PATH_INFO: `/v3/projects/8e9d4ca15763492bad22165413bae2bd` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:27.755849 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6604ac45-2b80-4b02-a795-27a8bb08e7d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e9d4ca15763492bad22165413bae2bd)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:27.758064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6604ac45-2b80-4b02-a795-27a8bb08e7d7 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:27.848163 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6604ac45-2b80-4b02-a795-27a8bb08e7d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e9d4ca15763492bad22165413bae2bd', 'request_id': 'req-6604ac45-2b80-4b02-a795-27a8bb08e7d7'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:27.859421 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6604ac45-2b80-4b02-a795-27a8bb08e7d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e9d4ca15763492bad22165413bae2bd', 'request_id': 'req-6604ac45-2b80-4b02-a795-27a8bb08e7d7'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:27.867976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6604ac45-2b80-4b02-a795-27a8bb08e7d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e9d4ca15763492bad22165413bae2bd', 'request_id': 'req-6604ac45-2b80-4b02-a795-27a8bb08e7d7'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:27.889010 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6604ac45-2b80-4b02-a795-27a8bb08e7d7 admin admin] The token cache is being invalidate because project 8e9d4ca15763492bad22165413bae2bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:36:27.889782 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6604ac45-2b80-4b02-a795-27a8bb08e7d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:27.890896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6604ac45-2b80-4b02-a795-27a8bb08e7d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:27.893134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1158/2294] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:36:27 2022] DELETE /identity/v3/projects/8e9d4ca15763492bad22165413bae2bd => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:27.900301 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ca022-03e4-41ab-8eb0-acb780f5db23 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:27.901072 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ca022-03e4-41ab-8eb0-acb780f5db23 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:27.901072 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ca022-03e4-41ab-8eb0-acb780f5db23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:27.925188 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-765ca022-03e4-41ab-8eb0-acb780f5db23 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:27.926464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-765ca022-03e4-41ab-8eb0-acb780f5db23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:28.013317 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1137/2295] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:27 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:28.022065 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4fc8d-ccf3-4769-8387-c05f09a5cb88 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:28.022321 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4fc8d-ccf3-4769-8387-c05f09a5cb88 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:28.022741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4fc8d-ccf3-4769-8387-c05f09a5cb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:28.053270 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-b7e4fc8d-ccf3-4769-8387-c05f09a5cb88 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:28.054568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-b7e4fc8d-ccf3-4769-8387-c05f09a5cb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:28.064696 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1159/2296] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:28.074900 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ca022-03e4-41ab-8eb0-acb780f5db23 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:28.088810 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ca022-03e4-41ab-8eb0-acb780f5db23 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:28.095720 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bbf25d-a5f3-4882-b610-8ac6c75abaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:28.096574 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbf25d-a5f3-4882-b610-8ac6c75abaae admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:28.097026 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbf25d-a5f3-4882-b610-8ac6c75abaae admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:28.097383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbf25d-a5f3-4882-b610-8ac6c75abaae admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:28.098210 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bbf25d-a5f3-4882-b610-8ac6c75abaae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:36:28.098595 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bbf25d-a5f3-4882-b610-8ac6c75abaae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:28.100444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bbf25d-a5f3-4882-b610-8ac6c75abaae admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:28.119435 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1138/2297] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:36:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:36:28.125722 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4fc8d-ccf3-4769-8387-c05f09a5cb88 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:28.144614 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4fc8d-ccf3-4769-8387-c05f09a5cb88 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:28.152613 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c90c5-2c2f-408d-92cd-223cf7bc9fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:28.153594 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c90c5-2c2f-408d-92cd-223cf7bc9fa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:28.154060 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c90c5-2c2f-408d-92cd-223cf7bc9fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:28.154332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c90c5-2c2f-408d-92cd-223cf7bc9fa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:28.155058 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4c90c5-2c2f-408d-92cd-223cf7bc9fa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:28.157577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4c90c5-2c2f-408d-92cd-223cf7bc9fa3 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:28.226970 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1160/2298] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:36:28 2022] POST /identity/v3/projects => generated 384 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:36:28.228845 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bbf25d-a5f3-4882-b610-8ac6c75abaae admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:28.252339 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bbf25d-a5f3-4882-b610-8ac6c75abaae admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:28.351850 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf6083-8de7-4b2b-9e63-dca20e4a71c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:28.352557 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf6083-8de7-4b2b-9e63-dca20e4a71c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:28.352846 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf6083-8de7-4b2b-9e63-dca20e4a71c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:28.353975 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf6083-8de7-4b2b-9e63-dca20e4a71c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:28.353975 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf6083-8de7-4b2b-9e63-dca20e4a71c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:28.355824 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf6083-8de7-4b2b-9e63-dca20e4a71c3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:28.394599 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1139/2299] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:36:28 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:36:28.399854 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c90c5-2c2f-408d-92cd-223cf7bc9fa3 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:28.411836 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c90c5-2c2f-408d-92cd-223cf7bc9fa3 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:28.418613 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2515d7bf-b574-4177-bcc5-05e3c7491610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:28.419394 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515d7bf-b574-4177-bcc5-05e3c7491610 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:28.419783 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515d7bf-b574-4177-bcc5-05e3c7491610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:28.420279 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515d7bf-b574-4177-bcc5-05e3c7491610 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:28.421051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2515d7bf-b574-4177-bcc5-05e3c7491610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:28.422794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2515d7bf-b574-4177-bcc5-05e3c7491610 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:28.436179 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1161/2300] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:36:28 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:36:28.440507 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf6083-8de7-4b2b-9e63-dca20e4a71c3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:28.452139 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf6083-8de7-4b2b-9e63-dca20e4a71c3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:28.458732 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6151f76-cd20-4fb5-b23f-3363262e99d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:28.459590 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6151f76-cd20-4fb5-b23f-3363262e99d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:28.460011 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6151f76-cd20-4fb5-b23f-3363262e99d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:28.460416 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6151f76-cd20-4fb5-b23f-3363262e99d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:28.461317 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6151f76-cd20-4fb5-b23f-3363262e99d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:28.463097 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6151f76-cd20-4fb5-b23f-3363262e99d9 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:28.478689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1140/2301] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:36:28 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:36:28.484521 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2515d7bf-b574-4177-bcc5-05e3c7491610 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:28.495757 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2515d7bf-b574-4177-bcc5-05e3c7491610 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:28.505490 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abaca4-446a-49d8-94eb-a494a4aaed90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:28.506205 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abaca4-446a-49d8-94eb-a494a4aaed90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:28.506495 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abaca4-446a-49d8-94eb-a494a4aaed90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:28.506777 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abaca4-446a-49d8-94eb-a494a4aaed90 admin admin] PATH_INFO: `/v3/projects/aacec5c4a2584f60af1ca9843e9e8732/users/db9e00718840481ca416e25ef436bb24/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:28.528540 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1abaca4-446a-49d8-94eb-a494a4aaed90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aacec5c4a2584f60af1ca9843e9e8732, user_id=db9e00718840481ca416e25ef436bb24, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:28.531425 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1abaca4-446a-49d8-94eb-a494a4aaed90 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:28.559403 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1162/2302] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:36:28 2022] PUT /identity/v3/projects/aacec5c4a2584f60af1ca9843e9e8732/users/db9e00718840481ca416e25ef436bb24/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:28.560128 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b83149-1a09-4454-b7a9-00c5a6b7a442 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:28.560432 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b83149-1a09-4454-b7a9-00c5a6b7a442 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:28.560721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b83149-1a09-4454-b7a9-00c5a6b7a442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:28.595939 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-31b83149-1a09-4454-b7a9-00c5a6b7a442 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:28.597753 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-31b83149-1a09-4454-b7a9-00c5a6b7a442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:28.637683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:28.708854 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1141/2303] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:28.722449 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abaca4-446a-49d8-94eb-a494a4aaed90 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:28.725207 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-f1abaca4-446a-49d8-94eb-a494a4aaed90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:28.806646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:28.838143 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abaca4-446a-49d8-94eb-a494a4aaed90 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:28.846433 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f3484-e26f-4965-9939-18f354348d7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:28.847847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f3484-e26f-4965-9939-18f354348d7c service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:28.848629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f3484-e26f-4965-9939-18f354348d7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:28.849357 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f3484-e26f-4965-9939-18f354348d7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:28.863922 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f3484-e26f-4965-9939-18f354348d7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:28.865457 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f3484-e26f-4965-9939-18f354348d7c service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:28.880712 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1163/2304] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:36:28 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:36:29.809937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8991e1-8a7c-4220-8615-6e29737c9d6b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:29.810722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8991e1-8a7c-4220-8615-6e29737c9d6b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:29.810722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8991e1-8a7c-4220-8615-6e29737c9d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:29.850149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9e8991e1-8a7c-4220-8615-6e29737c9d6b None None] MFA Rules not processed for user `db9e00718840481ca416e25ef436bb24`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:29.851861 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9e8991e1-8a7c-4220-8615-6e29737c9d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:29.937348 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1142/2305] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:29.945686 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f3484-e26f-4965-9939-18f354348d7c service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:29.947937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-923f3484-e26f-4965-9939-18f354348d7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:30.028663 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f3484-e26f-4965-9939-18f354348d7c service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:30.032956 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6acf89-1e12-4055-8c33-04399f350e1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:30.033720 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6acf89-1e12-4055-8c33-04399f350e1b service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:30.033911 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6acf89-1e12-4055-8c33-04399f350e1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:30.034193 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6acf89-1e12-4055-8c33-04399f350e1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:30.044231 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6acf89-1e12-4055-8c33-04399f350e1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:30.045729 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6acf89-1e12-4055-8c33-04399f350e1b service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:30.056140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1164/2306] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:36:29 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:36:32.637241 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762fbed-69d7-400f-86cc-cbe243e1583b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:32.638178 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762fbed-69d7-400f-86cc-cbe243e1583b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:32.638178 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762fbed-69d7-400f-86cc-cbe243e1583b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:32.660483 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6762fbed-69d7-400f-86cc-cbe243e1583b None None] MFA Rules not processed for user `db9e00718840481ca416e25ef436bb24`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:32.661776 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6762fbed-69d7-400f-86cc-cbe243e1583b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:32.669393 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1143/2307] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:32 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:32.895073 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd68d9-313a-46a8-930b-5cec555d582b None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:32.895883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd68d9-313a-46a8-930b-5cec555d582b None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:32.895883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd68d9-313a-46a8-930b-5cec555d582b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:32.919919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-bbcd68d9-313a-46a8-930b-5cec555d582b None None] MFA Rules not processed for user `efe6e59228bd49bfbff25a11eb682426`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:32.921332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-bbcd68d9-313a-46a8-930b-5cec555d582b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:32.987896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1165/2308] 10.222.0.121 () {58 vars in 1014 bytes} [Thu Aug 4 10:36:32 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:33.052111 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762fbed-69d7-400f-86cc-cbe243e1583b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:33.054330 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6762fbed-69d7-400f-86cc-cbe243e1583b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:33.157285 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762fbed-69d7-400f-86cc-cbe243e1583b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:33.164682 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611346a-7e7d-4f79-a99c-f8dc53bcade3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e16e26784601425c9883acd91f65becc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:33.165357 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611346a-7e7d-4f79-a99c-f8dc53bcade3 service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:33.165643 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611346a-7e7d-4f79-a99c-f8dc53bcade3 service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:33.165925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611346a-7e7d-4f79-a99c-f8dc53bcade3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:33.180433 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9611346a-7e7d-4f79-a99c-f8dc53bcade3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:33.183462 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9611346a-7e7d-4f79-a99c-f8dc53bcade3 service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:33.195609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1144/2309] 10.222.0.121 () {60 vars in 1389 bytes} [Thu Aug 4 10:36:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:36:42.980389 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=60 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:43.191026 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=81 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:51.741465 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7501e93-5d5d-444f-93d8-ee99ed18f568 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:51.742158 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7501e93-5d5d-444f-93d8-ee99ed18f568 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:51.742158 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7501e93-5d5d-444f-93d8-ee99ed18f568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:51.768611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-b7501e93-5d5d-444f-93d8-ee99ed18f568 None None] MFA Rules not processed for user `efe6e59228bd49bfbff25a11eb682426`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:51.769936 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-b7501e93-5d5d-444f-93d8-ee99ed18f568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:51.777527 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1166/2310] 10.222.0.121 () {58 vars in 972 bytes} [Thu Aug 4 10:36:51 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:51.791603 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611346a-7e7d-4f79-a99c-f8dc53bcade3 service swift] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:51.804309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611346a-7e7d-4f79-a99c-f8dc53bcade3 service swift] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:51.809746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c70a9f-59e0-4596-89b2-097cb83faa87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e16e26784601425c9883acd91f65becc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:51.810398 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c70a9f-59e0-4596-89b2-097cb83faa87 service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:51.810677 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c70a9f-59e0-4596-89b2-097cb83faa87 service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:51.810947 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c70a9f-59e0-4596-89b2-097cb83faa87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:51.823345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c70a9f-59e0-4596-89b2-097cb83faa87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:51.826223 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c70a9f-59e0-4596-89b2-097cb83faa87 service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:51.837068 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1145/2311] 10.222.0.121 () {60 vars in 1389 bytes} [Thu Aug 4 10:36:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:36:54.250348 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7501e93-5d5d-444f-93d8-ee99ed18f568 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:54.252976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-b7501e93-5d5d-444f-93d8-ee99ed18f568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:54.280833 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7501e93-5d5d-444f-93d8-ee99ed18f568 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:54.287254 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15af65-3284-4d79-bec2-9f461e89a257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:54.293200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15af65-3284-4d79-bec2-9f461e89a257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:54.293611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15af65-3284-4d79-bec2-9f461e89a257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:54.294032 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15af65-3284-4d79-bec2-9f461e89a257 admin admin] PATH_INFO: `/v3/users/a025ea88db864b55b7537311d20f4808` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:54.297164 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15af65-3284-4d79-bec2-9f461e89a257 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a025ea88db864b55b7537311d20f4808)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:54.299656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15af65-3284-4d79-bec2-9f461e89a257 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:54.433619 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0e15af65-3284-4d79-bec2-9f461e89a257 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a025ea88db864b55b7537311d20f4808'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:54.434763 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0e15af65-3284-4d79-bec2-9f461e89a257 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a025ea88db864b55b7537311d20f4808'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:54.463291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0e15af65-3284-4d79-bec2-9f461e89a257 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a025ea88db864b55b7537311d20f4808'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:54.482774 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0e15af65-3284-4d79-bec2-9f461e89a257 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a025ea88db864b55b7537311d20f4808'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:54.499372 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-0e15af65-3284-4d79-bec2-9f461e89a257 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a025ea88db864b55b7537311d20f4808'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:54.513919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1167/2312] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:36:54 2022] DELETE /identity/v3/users/a025ea88db864b55b7537311d20f4808 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:54.824538 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c70a9f-59e0-4596-89b2-097cb83faa87 service swift] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:54.843418 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c70a9f-59e0-4596-89b2-097cb83faa87 service swift] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:54.945505 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d12924-2e00-4c0a-987a-6a12e7530c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:54.946301 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d12924-2e00-4c0a-987a-6a12e7530c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:54.946655 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d12924-2e00-4c0a-987a-6a12e7530c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:54.946977 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d12924-2e00-4c0a-987a-6a12e7530c8b admin admin] PATH_INFO: `/v3/projects/99b1be624402454ba326a7d7a8754a25` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:54.948510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d12924-2e00-4c0a-987a-6a12e7530c8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=99b1be624402454ba326a7d7a8754a25)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:54.950852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d12924-2e00-4c0a-987a-6a12e7530c8b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:55.025185 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f5d12924-2e00-4c0a-987a-6a12e7530c8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99b1be624402454ba326a7d7a8754a25', 'request_id': 'req-f5d12924-2e00-4c0a-987a-6a12e7530c8b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:55.040173 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f5d12924-2e00-4c0a-987a-6a12e7530c8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99b1be624402454ba326a7d7a8754a25', 'request_id': 'req-f5d12924-2e00-4c0a-987a-6a12e7530c8b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:55.050897 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f5d12924-2e00-4c0a-987a-6a12e7530c8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99b1be624402454ba326a7d7a8754a25', 'request_id': 'req-f5d12924-2e00-4c0a-987a-6a12e7530c8b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:55.071524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f5d12924-2e00-4c0a-987a-6a12e7530c8b admin admin] The token cache is being invalidate because project 99b1be624402454ba326a7d7a8754a25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:36:55.072023 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f5d12924-2e00-4c0a-987a-6a12e7530c8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:55.072919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f5d12924-2e00-4c0a-987a-6a12e7530c8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:36:55.074874 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1146/2313] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:36:54 2022] DELETE /identity/v3/projects/99b1be624402454ba326a7d7a8754a25 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:55.081696 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7be7c7-c7b1-4c6f-b49e-9b00590d459c None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:55.082200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7be7c7-c7b1-4c6f-b49e-9b00590d459c None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:55.082712 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7be7c7-c7b1-4c6f-b49e-9b00590d459c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:55.125958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-ca7be7c7-c7b1-4c6f-b49e-9b00590d459c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:55.129338 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-ca7be7c7-c7b1-4c6f-b49e-9b00590d459c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:55.264180 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1168/2314] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:55 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:55.271878 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29f629-a31c-490d-9265-be6b384bbc2e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:55.271878 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29f629-a31c-490d-9265-be6b384bbc2e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:55.271878 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29f629-a31c-490d-9265-be6b384bbc2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:55.299566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8c29f629-a31c-490d-9265-be6b384bbc2e None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:55.301465 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8c29f629-a31c-490d-9265-be6b384bbc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:55.312053 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1147/2315] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:55 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:55.317292 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7be7c7-c7b1-4c6f-b49e-9b00590d459c None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:55.328782 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7be7c7-c7b1-4c6f-b49e-9b00590d459c None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:55.334677 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe35d2-1a99-4551-8076-be367395d432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:55.335341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe35d2-1a99-4551-8076-be367395d432 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:55.335619 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe35d2-1a99-4551-8076-be367395d432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:55.335896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe35d2-1a99-4551-8076-be367395d432 admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:55.336753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe35d2-1a99-4551-8076-be367395d432 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:36:55.337066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe35d2-1a99-4551-8076-be367395d432 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:55.338751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe35d2-1a99-4551-8076-be367395d432 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:55.367203 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1169/2316] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:36:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:36:55.368750 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29f629-a31c-490d-9265-be6b384bbc2e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:55.388107 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29f629-a31c-490d-9265-be6b384bbc2e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:55.394386 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39020122-ec7d-49f8-bcca-af9764deaf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:55.395163 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39020122-ec7d-49f8-bcca-af9764deaf3e admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:55.395385 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39020122-ec7d-49f8-bcca-af9764deaf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:55.395642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39020122-ec7d-49f8-bcca-af9764deaf3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:55.396478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39020122-ec7d-49f8-bcca-af9764deaf3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:55.398508 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39020122-ec7d-49f8-bcca-af9764deaf3e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:55.435320 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1148/2317] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:36:55 2022] POST /identity/v3/projects => generated 400 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:36:55.441140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe35d2-1a99-4551-8076-be367395d432 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:55.460236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe35d2-1a99-4551-8076-be367395d432 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:55.564124 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:55.582608 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f590e-5730-49b0-ba7c-fedc6a3bfcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:55.583579 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f590e-5730-49b0-ba7c-fedc6a3bfcc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:55.584157 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f590e-5730-49b0-ba7c-fedc6a3bfcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:55.584588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f590e-5730-49b0-ba7c-fedc6a3bfcc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:55.585715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0f590e-5730-49b0-ba7c-fedc6a3bfcc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:55.588551 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0f590e-5730-49b0-ba7c-fedc6a3bfcc4 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:55.618123 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1170/2318] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:36:55 2022] POST /identity/v3/users => generated 301 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:36:55.623576 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39020122-ec7d-49f8-bcca-af9764deaf3e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:55.641973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39020122-ec7d-49f8-bcca-af9764deaf3e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:55.648221 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3246cfd-1fce-4c62-b16e-139cb5144b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:55.648886 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3246cfd-1fce-4c62-b16e-139cb5144b58 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:55.649170 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3246cfd-1fce-4c62-b16e-139cb5144b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:55.650333 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3246cfd-1fce-4c62-b16e-139cb5144b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:55.650333 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3246cfd-1fce-4c62-b16e-139cb5144b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:55.652117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3246cfd-1fce-4c62-b16e-139cb5144b58 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:55.666990 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1149/2319] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:36:55 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:36:55.671148 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f590e-5730-49b0-ba7c-fedc6a3bfcc4 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:55.682849 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f590e-5730-49b0-ba7c-fedc6a3bfcc4 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:55.689337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c772f4c9-ff35-4503-8e99-7bf26dd58573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:55.690012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772f4c9-ff35-4503-8e99-7bf26dd58573 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:55.690356 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772f4c9-ff35-4503-8e99-7bf26dd58573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:55.690622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772f4c9-ff35-4503-8e99-7bf26dd58573 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:55.691364 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c772f4c9-ff35-4503-8e99-7bf26dd58573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:55.693293 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c772f4c9-ff35-4503-8e99-7bf26dd58573 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:55.716304 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1171/2320] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:36:55 2022] GET /identity/v3/roles => generated 1427 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:36:55.720891 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3246cfd-1fce-4c62-b16e-139cb5144b58 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:55.741599 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3246cfd-1fce-4c62-b16e-139cb5144b58 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:55.754024 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e504c7-5286-4e4b-bf06-f623e769df0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:55.754024 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e504c7-5286-4e4b-bf06-f623e769df0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:55.754444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e504c7-5286-4e4b-bf06-f623e769df0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:55.754444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e504c7-5286-4e4b-bf06-f623e769df0b admin admin] PATH_INFO: `/v3/projects/c9be721641884e34a8b6b98b434ec3d8/users/190fa8caf4b243e98e1be4f18f5b52ca/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:55.781717 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e504c7-5286-4e4b-bf06-f623e769df0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9be721641884e34a8b6b98b434ec3d8, user_id=190fa8caf4b243e98e1be4f18f5b52ca, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:55.785171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e504c7-5286-4e4b-bf06-f623e769df0b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:55.804990 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1150/2321] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:36:55 2022] PUT /identity/v3/projects/c9be721641884e34a8b6b98b434ec3d8/users/190fa8caf4b243e98e1be4f18f5b52ca/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:36:55.821010 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ddc50f-d2e2-497c-85ef-4c3a0066b857 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:55.821367 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ddc50f-d2e2-497c-85ef-4c3a0066b857 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:55.821742 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ddc50f-d2e2-497c-85ef-4c3a0066b857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:55.888528 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-11ddc50f-d2e2-497c-85ef-4c3a0066b857 None None] MFA Rules not processed for user `190fa8caf4b243e98e1be4f18f5b52ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:55.892654 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-11ddc50f-d2e2-497c-85ef-4c3a0066b857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:56.047263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1172/2322] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:36:55 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:36:56.085114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e504c7-5286-4e4b-bf06-f623e769df0b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:56.089090 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-69e504c7-5286-4e4b-bf06-f623e769df0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:56.122874 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:36:56.267213 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e504c7-5286-4e4b-bf06-f623e769df0b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:56.272838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b8c5c-b6ea-4f0f-a0ed-c4bf15b23f71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:56.273851 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b8c5c-b6ea-4f0f-a0ed-c4bf15b23f71 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:56.274143 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b8c5c-b6ea-4f0f-a0ed-c4bf15b23f71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:56.274416 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b8c5c-b6ea-4f0f-a0ed-c4bf15b23f71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:56.290620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b8c5c-b6ea-4f0f-a0ed-c4bf15b23f71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:56.292794 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b8c5c-b6ea-4f0f-a0ed-c4bf15b23f71 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:56.307258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1151/2323] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:36:56 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:36:59.144123 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed73820-8785-4dcb-80a1-078a14adac34 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:59.144123 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed73820-8785-4dcb-80a1-078a14adac34 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:59.144123 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed73820-8785-4dcb-80a1-078a14adac34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:59.167332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-7ed73820-8785-4dcb-80a1-078a14adac34 None None] MFA Rules not processed for user `efe6e59228bd49bfbff25a11eb682426`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:36:59.168745 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-7ed73820-8785-4dcb-80a1-078a14adac34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:59.264304 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1173/2324] 10.222.0.121 () {58 vars in 1014 bytes} [Thu Aug 4 10:36:59 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:36:59.277328 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b8c5c-b6ea-4f0f-a0ed-c4bf15b23f71 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:36:59.280188 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a91b8c5c-b6ea-4f0f-a0ed-c4bf15b23f71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:36:59.385428 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b8c5c-b6ea-4f0f-a0ed-c4bf15b23f71 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:36:59.393354 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22029f4-d5a6-4660-a842-f611a434ac8a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e16e26784601425c9883acd91f65becc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:36:59.394049 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22029f4-d5a6-4660-a842-f611a434ac8a service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:36:59.394369 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22029f4-d5a6-4660-a842-f611a434ac8a service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:36:59.394753 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22029f4-d5a6-4660-a842-f611a434ac8a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:36:59.410251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22029f4-d5a6-4660-a842-f611a434ac8a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:36:59.413436 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22029f4-d5a6-4660-a842-f611a434ac8a service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:36:59.425968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1152/2325] 10.222.0.121 () {60 vars in 1389 bytes} [Thu Aug 4 10:36:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:37:00.077657 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01587871-d588-41f4-825f-d36da214154a None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:00.078248 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01587871-d588-41f4-825f-d36da214154a None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:00.078248 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01587871-d588-41f4-825f-d36da214154a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:00.100220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-01587871-d588-41f4-825f-d36da214154a None None] MFA Rules not processed for user `efe6e59228bd49bfbff25a11eb682426`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:37:00.101822 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-01587871-d588-41f4-825f-d36da214154a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:00.112462 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1174/2326] 10.222.0.121 () {58 vars in 972 bytes} [Thu Aug 4 10:37:00 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:37:00.129565 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22029f4-d5a6-4660-a842-f611a434ac8a service swift] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:00.141862 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22029f4-d5a6-4660-a842-f611a434ac8a service swift] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:00.147782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42474947-2bc3-4c5b-b2c1-3787c58a9127 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e16e26784601425c9883acd91f65becc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:00.148584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42474947-2bc3-4c5b-b2c1-3787c58a9127 service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:00.149032 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42474947-2bc3-4c5b-b2c1-3787c58a9127 service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:00.149508 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42474947-2bc3-4c5b-b2c1-3787c58a9127 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:00.162393 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42474947-2bc3-4c5b-b2c1-3787c58a9127 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:00.164172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42474947-2bc3-4c5b-b2c1-3787c58a9127 service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:00.173810 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1153/2327] 10.222.0.121 () {60 vars in 1389 bytes} [Thu Aug 4 10:37:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:37:00.457196 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49285d7a-c776-414e-a503-1dc329a1f947 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:00.457954 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49285d7a-c776-414e-a503-1dc329a1f947 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:00.457954 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49285d7a-c776-414e-a503-1dc329a1f947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:00.483158 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-49285d7a-c776-414e-a503-1dc329a1f947 None None] MFA Rules not processed for user `efe6e59228bd49bfbff25a11eb682426`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:37:00.484348 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-49285d7a-c776-414e-a503-1dc329a1f947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:00.492983 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1175/2328] 10.222.0.121 () {58 vars in 1014 bytes} [Thu Aug 4 10:37:00 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:37:00.509477 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42474947-2bc3-4c5b-b2c1-3787c58a9127 service swift] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:00.522179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42474947-2bc3-4c5b-b2c1-3787c58a9127 service swift] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:00.528582 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc810c8-26f2-4f7e-a052-8cfe54b828d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e16e26784601425c9883acd91f65becc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:00.529251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc810c8-26f2-4f7e-a052-8cfe54b828d8 service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:00.529538 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc810c8-26f2-4f7e-a052-8cfe54b828d8 service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:00.529816 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc810c8-26f2-4f7e-a052-8cfe54b828d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:00.540695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc810c8-26f2-4f7e-a052-8cfe54b828d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:00.543361 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc810c8-26f2-4f7e-a052-8cfe54b828d8 service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:00.557224 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1154/2329] 10.222.0.121 () {60 vars in 1389 bytes} [Thu Aug 4 10:37:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:37:00.965589 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fc94e-55b3-49db-8373-01bfd55b1e3a None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:00.965879 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fc94e-55b3-49db-8373-01bfd55b1e3a None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:00.966161 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fc94e-55b3-49db-8373-01bfd55b1e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:00.988678 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-7f2fc94e-55b3-49db-8373-01bfd55b1e3a None None] MFA Rules not processed for user `efe6e59228bd49bfbff25a11eb682426`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:37:00.990001 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-7f2fc94e-55b3-49db-8373-01bfd55b1e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:00.997701 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1176/2330] 10.222.0.121 () {58 vars in 972 bytes} [Thu Aug 4 10:37:00 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:37:01.010905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc810c8-26f2-4f7e-a052-8cfe54b828d8 service swift] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:01.023522 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc810c8-26f2-4f7e-a052-8cfe54b828d8 service swift] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:01.031319 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef132b22-ebb3-4268-bb44-932b25e90c29 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e16e26784601425c9883acd91f65becc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:01.032237 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef132b22-ebb3-4268-bb44-932b25e90c29 service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:01.032573 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef132b22-ebb3-4268-bb44-932b25e90c29 service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:01.032868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef132b22-ebb3-4268-bb44-932b25e90c29 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:01.044169 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef132b22-ebb3-4268-bb44-932b25e90c29 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:01.045874 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef132b22-ebb3-4268-bb44-932b25e90c29 service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:01.055514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1155/2331] 10.222.0.121 () {60 vars in 1389 bytes} [Thu Aug 4 10:37:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:37:05.580906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fc94e-55b3-49db-8373-01bfd55b1e3a None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:05.582788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-7f2fc94e-55b3-49db-8373-01bfd55b1e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:05.648811 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:37:05.678201 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fc94e-55b3-49db-8373-01bfd55b1e3a None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:05.686066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0425c-4bb5-4c37-a2fd-a3193bea244c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:05.686788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0425c-4bb5-4c37-a2fd-a3193bea244c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:05.687067 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0425c-4bb5-4c37-a2fd-a3193bea244c admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:05.687345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0425c-4bb5-4c37-a2fd-a3193bea244c admin admin] PATH_INFO: `/v3/users/c06eaea526a346b3b86f27eb347b9856` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:05.689070 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0425c-4bb5-4c37-a2fd-a3193bea244c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c06eaea526a346b3b86f27eb347b9856)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:05.692129 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0425c-4bb5-4c37-a2fd-a3193bea244c admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:05.772193 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-52c0425c-4bb5-4c37-a2fd-a3193bea244c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c06eaea526a346b3b86f27eb347b9856'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:05.773210 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-52c0425c-4bb5-4c37-a2fd-a3193bea244c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c06eaea526a346b3b86f27eb347b9856'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:05.797922 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-52c0425c-4bb5-4c37-a2fd-a3193bea244c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c06eaea526a346b3b86f27eb347b9856'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:05.810276 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-52c0425c-4bb5-4c37-a2fd-a3193bea244c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c06eaea526a346b3b86f27eb347b9856'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:05.828478 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-52c0425c-4bb5-4c37-a2fd-a3193bea244c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c06eaea526a346b3b86f27eb347b9856'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:05.838256 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1177/2332] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:37:05 2022] DELETE /identity/v3/users/c06eaea526a346b3b86f27eb347b9856 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:37:05.843238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef167e01-6f18-4564-8208-673f644ab375 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:05.844633 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef167e01-6f18-4564-8208-673f644ab375 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:05.845437 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef167e01-6f18-4564-8208-673f644ab375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:05.868796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ef167e01-6f18-4564-8208-673f644ab375 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:37:05.870120 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ef167e01-6f18-4564-8208-673f644ab375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:05.990375 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1156/2333] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:37:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:37:06.006422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0425c-4bb5-4c37-a2fd-a3193bea244c admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:06.006831 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-52c0425c-4bb5-4c37-a2fd-a3193bea244c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:06.144448 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0425c-4bb5-4c37-a2fd-a3193bea244c admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:06.151464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f23cc-6c5e-4b6e-ae42-070b712b6150 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:06.152752 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f23cc-6c5e-4b6e-ae42-070b712b6150 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:06.153268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f23cc-6c5e-4b6e-ae42-070b712b6150 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:06.153624 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f23cc-6c5e-4b6e-ae42-070b712b6150 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:06.171869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f23cc-6c5e-4b6e-ae42-070b712b6150 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:06.173396 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f23cc-6c5e-4b6e-ae42-070b712b6150 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:06.192888 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1178/2334] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:37:06 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:37:06.437927 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef167e01-6f18-4564-8208-673f644ab375 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:06.439861 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ef167e01-6f18-4564-8208-673f644ab375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:06.459371 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef167e01-6f18-4564-8208-673f644ab375 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:06.467908 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0ee6d-925c-45a4-a61f-cdc054ce0fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:06.468836 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0ee6d-925c-45a4-a61f-cdc054ce0fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:06.469081 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0ee6d-925c-45a4-a61f-cdc054ce0fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:06.469406 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0ee6d-925c-45a4-a61f-cdc054ce0fe2 admin admin] PATH_INFO: `/v3/projects/3c41c04883804e9586e241166703c1ce` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:06.470888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0ee6d-925c-45a4-a61f-cdc054ce0fe2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c41c04883804e9586e241166703c1ce)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:06.474004 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0ee6d-925c-45a4-a61f-cdc054ce0fe2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:06.477630 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:37:06.598788 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2ab0ee6d-925c-45a4-a61f-cdc054ce0fe2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c41c04883804e9586e241166703c1ce', 'request_id': 'req-2ab0ee6d-925c-45a4-a61f-cdc054ce0fe2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:06.609607 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2ab0ee6d-925c-45a4-a61f-cdc054ce0fe2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c41c04883804e9586e241166703c1ce', 'request_id': 'req-2ab0ee6d-925c-45a4-a61f-cdc054ce0fe2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:06.623658 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2ab0ee6d-925c-45a4-a61f-cdc054ce0fe2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c41c04883804e9586e241166703c1ce', 'request_id': 'req-2ab0ee6d-925c-45a4-a61f-cdc054ce0fe2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:06.648998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2ab0ee6d-925c-45a4-a61f-cdc054ce0fe2 admin admin] The token cache is being invalidate because project 3c41c04883804e9586e241166703c1ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:37:06.649369 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2ab0ee6d-925c-45a4-a61f-cdc054ce0fe2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:06.650906 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2ab0ee6d-925c-45a4-a61f-cdc054ce0fe2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:06.654301 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1157/2335] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:37:06 2022] DELETE /identity/v3/projects/3c41c04883804e9586e241166703c1ce => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:37:06.661931 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72416a3-5b82-41f1-8a04-9627cb581291 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:06.663046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72416a3-5b82-41f1-8a04-9627cb581291 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:06.663046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72416a3-5b82-41f1-8a04-9627cb581291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:06.697255 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-f72416a3-5b82-41f1-8a04-9627cb581291 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:37:06.698480 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-f72416a3-5b82-41f1-8a04-9627cb581291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:06.834144 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1179/2336] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:37:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:37:06.837113 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3e6ef-cdba-403b-a1f6-c0f310878c49 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:06.837407 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3e6ef-cdba-403b-a1f6-c0f310878c49 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:06.837718 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3e6ef-cdba-403b-a1f6-c0f310878c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:06.863996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7bf3e6ef-cdba-403b-a1f6-c0f310878c49 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:37:06.866170 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7bf3e6ef-cdba-403b-a1f6-c0f310878c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:06.874981 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1158/2337] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:37:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:37:06.880132 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72416a3-5b82-41f1-8a04-9627cb581291 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:06.891401 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72416a3-5b82-41f1-8a04-9627cb581291 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:06.898260 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc71b3-42f2-4f09-9b4b-b1fe90191d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:06.898937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc71b3-42f2-4f09-9b4b-b1fe90191d9c admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:06.899231 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc71b3-42f2-4f09-9b4b-b1fe90191d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:06.899512 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc71b3-42f2-4f09-9b4b-b1fe90191d9c admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:06.900303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc71b3-42f2-4f09-9b4b-b1fe90191d9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:37:06.900607 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc71b3-42f2-4f09-9b4b-b1fe90191d9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:06.902301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc71b3-42f2-4f09-9b4b-b1fe90191d9c admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:06.905601 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:37:06.930650 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1180/2338] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:37:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:37:06.932223 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3e6ef-cdba-403b-a1f6-c0f310878c49 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:06.944172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3e6ef-cdba-403b-a1f6-c0f310878c49 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:06.950001 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5bc5b5-d31e-4fdb-a755-f009d133f9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:06.950661 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5bc5b5-d31e-4fdb-a755-f009d133f9c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:06.950951 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5bc5b5-d31e-4fdb-a755-f009d133f9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:06.951369 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5bc5b5-d31e-4fdb-a755-f009d133f9c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:06.952459 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5bc5b5-d31e-4fdb-a755-f009d133f9c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:06.953960 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5bc5b5-d31e-4fdb-a755-f009d133f9c2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:06.993582 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1159/2339] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:37:06 2022] POST /identity/v3/projects => generated 390 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:37:06.998959 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc71b3-42f2-4f09-9b4b-b1fe90191d9c admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:07.012324 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc71b3-42f2-4f09-9b4b-b1fe90191d9c admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:07.114879 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cecb303-f0b1-4639-96bb-b2da386c1376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:07.114879 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecb303-f0b1-4639-96bb-b2da386c1376 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:07.114879 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecb303-f0b1-4639-96bb-b2da386c1376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:07.115485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecb303-f0b1-4639-96bb-b2da386c1376 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:07.115864 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cecb303-f0b1-4639-96bb-b2da386c1376 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:07.117943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cecb303-f0b1-4639-96bb-b2da386c1376 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:07.147089 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1181/2340] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:37:06 2022] POST /identity/v3/users => generated 296 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:37:07.153939 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5bc5b5-d31e-4fdb-a755-f009d133f9c2 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:07.169544 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5bc5b5-d31e-4fdb-a755-f009d133f9c2 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:07.177769 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228542c-fd32-4a96-8fa3-203d3cfd45c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:07.178430 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228542c-fd32-4a96-8fa3-203d3cfd45c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:07.178718 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228542c-fd32-4a96-8fa3-203d3cfd45c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:07.178980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228542c-fd32-4a96-8fa3-203d3cfd45c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:07.179698 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5228542c-fd32-4a96-8fa3-203d3cfd45c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:07.181414 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5228542c-fd32-4a96-8fa3-203d3cfd45c3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:07.194719 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1160/2341] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:37:07 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:37:07.197208 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cecb303-f0b1-4639-96bb-b2da386c1376 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:07.199500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228542c-fd32-4a96-8fa3-203d3cfd45c3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:07.202558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-3cecb303-f0b1-4639-96bb-b2da386c1376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:07.216175 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228542c-fd32-4a96-8fa3-203d3cfd45c3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:07.222356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d39903-e3d2-4dc1-8188-5bd4ff3eb20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:07.223028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d39903-e3d2-4dc1-8188-5bd4ff3eb20e admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:07.223312 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d39903-e3d2-4dc1-8188-5bd4ff3eb20e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:07.223587 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d39903-e3d2-4dc1-8188-5bd4ff3eb20e admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:07.224350 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d39903-e3d2-4dc1-8188-5bd4ff3eb20e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:07.229189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cecb303-f0b1-4639-96bb-b2da386c1376 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:07.229404 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d39903-e3d2-4dc1-8188-5bd4ff3eb20e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:07.236963 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685e7d8-4b5c-4ae1-a475-2c58e7aa592e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:07.237868 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685e7d8-4b5c-4ae1-a475-2c58e7aa592e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:07.238187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685e7d8-4b5c-4ae1-a475-2c58e7aa592e admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:07.238469 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685e7d8-4b5c-4ae1-a475-2c58e7aa592e admin admin] PATH_INFO: `/v3/users/190fa8caf4b243e98e1be4f18f5b52ca` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:07.241459 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685e7d8-4b5c-4ae1-a475-2c58e7aa592e admin admin] RBAC: Authorizing `identity:delete_user(user_id=190fa8caf4b243e98e1be4f18f5b52ca)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:07.245633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685e7d8-4b5c-4ae1-a475-2c58e7aa592e admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:07.253844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1161/2342] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:37:07 2022] GET /identity/v3/roles => generated 1427 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:37:07.259392 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d39903-e3d2-4dc1-8188-5bd4ff3eb20e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:07.275252 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d39903-e3d2-4dc1-8188-5bd4ff3eb20e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:07.284479 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a528bbd-1d47-4988-98a0-5ad3be94409c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:07.286023 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a528bbd-1d47-4988-98a0-5ad3be94409c admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:07.286774 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a528bbd-1d47-4988-98a0-5ad3be94409c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:07.288029 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a528bbd-1d47-4988-98a0-5ad3be94409c admin admin] PATH_INFO: `/v3/projects/39068c2e75774c41ab345bba771c00ea/users/30992f46b39740ef86af650859fb48d5/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:07.315205 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a528bbd-1d47-4988-98a0-5ad3be94409c admin admin] RBAC: Authorizing `identity:create_grant(project_id=39068c2e75774c41ab345bba771c00ea, user_id=30992f46b39740ef86af650859fb48d5, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:07.320165 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a528bbd-1d47-4988-98a0-5ad3be94409c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:07.343018 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1162/2343] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:37:07 2022] PUT /identity/v3/projects/39068c2e75774c41ab345bba771c00ea/users/30992f46b39740ef86af650859fb48d5/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:37:07.356692 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6dcb8-9830-43f9-9948-244bf10483c1 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:07.357182 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b685e7d8-4b5c-4ae1-a475-2c58e7aa592e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '190fa8caf4b243e98e1be4f18f5b52ca'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:07.357361 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6dcb8-9830-43f9-9948-244bf10483c1 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:07.358065 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6dcb8-9830-43f9-9948-244bf10483c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:07.362850 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b685e7d8-4b5c-4ae1-a475-2c58e7aa592e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '190fa8caf4b243e98e1be4f18f5b52ca'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:07.390876 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b685e7d8-4b5c-4ae1-a475-2c58e7aa592e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '190fa8caf4b243e98e1be4f18f5b52ca'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:07.406338 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b685e7d8-4b5c-4ae1-a475-2c58e7aa592e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '190fa8caf4b243e98e1be4f18f5b52ca'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:07.410661 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0cf6dcb8-9830-43f9-9948-244bf10483c1 None None] MFA Rules not processed for user `30992f46b39740ef86af650859fb48d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:37:07.411984 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0cf6dcb8-9830-43f9-9948-244bf10483c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:07.422084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b685e7d8-4b5c-4ae1-a475-2c58e7aa592e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '190fa8caf4b243e98e1be4f18f5b52ca'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:07.431031 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1182/2344] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:37:07 2022] DELETE /identity/v3/users/190fa8caf4b243e98e1be4f18f5b52ca => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:37:07.436097 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5501b-1719-4d46-aa42-b7866dd4ad9e None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:07.436481 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5501b-1719-4d46-aa42-b7866dd4ad9e None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:07.437731 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5501b-1719-4d46-aa42-b7866dd4ad9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:07.461173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-53d5501b-1719-4d46-aa42-b7866dd4ad9e None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:37:07.465502 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-53d5501b-1719-4d46-aa42-b7866dd4ad9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:07.518993 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1163/2345] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:37:07 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:37:07.535600 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6dcb8-9830-43f9-9948-244bf10483c1 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:07.537725 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0cf6dcb8-9830-43f9-9948-244bf10483c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:07.541734 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:37:07.545598 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:37:07.565934 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1183/2346] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:37:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:37:07.576668 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5501b-1719-4d46-aa42-b7866dd4ad9e None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:07.580388 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-53d5501b-1719-4d46-aa42-b7866dd4ad9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:07.627471 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6dcb8-9830-43f9-9948-244bf10483c1 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:07.670507 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5501b-1719-4d46-aa42-b7866dd4ad9e None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:07.675252 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5cb818-fb7f-49c1-a141-99ad1eb6de09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:07.675252 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5cb818-fb7f-49c1-a141-99ad1eb6de09 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:07.675777 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5cb818-fb7f-49c1-a141-99ad1eb6de09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:07.675777 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5cb818-fb7f-49c1-a141-99ad1eb6de09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:07.686681 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5cb818-fb7f-49c1-a141-99ad1eb6de09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:07.688519 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5cb818-fb7f-49c1-a141-99ad1eb6de09 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:07.703352 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1184/2347] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:37:07 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:37:07.714176 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5ec40-92ae-40ea-9d9b-b66116265902 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:07.714843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5ec40-92ae-40ea-9d9b-b66116265902 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:07.715123 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5ec40-92ae-40ea-9d9b-b66116265902 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:07.715530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5ec40-92ae-40ea-9d9b-b66116265902 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:07.825422 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5ec40-92ae-40ea-9d9b-b66116265902 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:07.826957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5ec40-92ae-40ea-9d9b-b66116265902 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:07.844749 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1164/2348] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:37:07 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:37:08.092522 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5cb818-fb7f-49c1-a141-99ad1eb6de09 service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:08.095369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6b5cb818-fb7f-49c1-a141-99ad1eb6de09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:08.129087 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5cb818-fb7f-49c1-a141-99ad1eb6de09 service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:08.135816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c81822-5caf-4932-aaf1-8e246efc6456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:08.136680 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c81822-5caf-4932-aaf1-8e246efc6456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:08.137058 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c81822-5caf-4932-aaf1-8e246efc6456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:08.137414 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c81822-5caf-4932-aaf1-8e246efc6456 admin admin] PATH_INFO: `/v3/projects/c9be721641884e34a8b6b98b434ec3d8` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:08.138992 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c81822-5caf-4932-aaf1-8e246efc6456 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9be721641884e34a8b6b98b434ec3d8)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:08.141608 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c81822-5caf-4932-aaf1-8e246efc6456 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:08.238258 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-66c81822-5caf-4932-aaf1-8e246efc6456 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9be721641884e34a8b6b98b434ec3d8', 'request_id': 'req-66c81822-5caf-4932-aaf1-8e246efc6456'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:08.254831 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-66c81822-5caf-4932-aaf1-8e246efc6456 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9be721641884e34a8b6b98b434ec3d8', 'request_id': 'req-66c81822-5caf-4932-aaf1-8e246efc6456'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:08.269345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-66c81822-5caf-4932-aaf1-8e246efc6456 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9be721641884e34a8b6b98b434ec3d8', 'request_id': 'req-66c81822-5caf-4932-aaf1-8e246efc6456'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:08.298902 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-66c81822-5caf-4932-aaf1-8e246efc6456 admin admin] The token cache is being invalidate because project c9be721641884e34a8b6b98b434ec3d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:37:08.299379 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-66c81822-5caf-4932-aaf1-8e246efc6456 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:08.300406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-66c81822-5caf-4932-aaf1-8e246efc6456 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:08.303918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1185/2349] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:37:08 2022] DELETE /identity/v3/projects/c9be721641884e34a8b6b98b434ec3d8 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:37:08.321395 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9978ac7-9a51-421b-87b4-6fe661e40da6 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:08.321395 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9978ac7-9a51-421b-87b4-6fe661e40da6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:08.321395 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9978ac7-9a51-421b-87b4-6fe661e40da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:08.348175 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e9978ac7-9a51-421b-87b4-6fe661e40da6 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:37:08.348992 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e9978ac7-9a51-421b-87b4-6fe661e40da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:08.462396 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1165/2350] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:37:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:37:08.477180 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6977d4-e6e6-4af7-be25-057b4e4bfb49 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:08.477654 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6977d4-e6e6-4af7-be25-057b4e4bfb49 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:08.478146 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6977d4-e6e6-4af7-be25-057b4e4bfb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:08.509922 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-9b6977d4-e6e6-4af7-be25-057b4e4bfb49 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:37:08.511258 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-9b6977d4-e6e6-4af7-be25-057b4e4bfb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:08.527102 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1186/2351] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:37:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:37:08.532008 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9978ac7-9a51-421b-87b4-6fe661e40da6 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:08.543281 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9978ac7-9a51-421b-87b4-6fe661e40da6 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:08.549236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda868e-d265-4ed4-8599-fecbcff9d7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:08.549931 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda868e-d265-4ed4-8599-fecbcff9d7be admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:08.550225 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda868e-d265-4ed4-8599-fecbcff9d7be admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:08.550501 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda868e-d265-4ed4-8599-fecbcff9d7be admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:08.551220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda868e-d265-4ed4-8599-fecbcff9d7be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:37:08.551536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda868e-d265-4ed4-8599-fecbcff9d7be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:08.553759 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda868e-d265-4ed4-8599-fecbcff9d7be admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:08.569030 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1166/2352] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:37:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:37:08.574297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6977d4-e6e6-4af7-be25-057b4e4bfb49 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:08.587152 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6977d4-e6e6-4af7-be25-057b4e4bfb49 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:08.593009 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b08ccf-0e92-4526-adf5-42a02b8bf3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:08.593671 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b08ccf-0e92-4526-adf5-42a02b8bf3db admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:08.593961 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b08ccf-0e92-4526-adf5-42a02b8bf3db admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:08.594243 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b08ccf-0e92-4526-adf5-42a02b8bf3db admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:08.594952 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b08ccf-0e92-4526-adf5-42a02b8bf3db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:08.597182 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b08ccf-0e92-4526-adf5-42a02b8bf3db admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:08.633891 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1187/2353] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:37:08 2022] POST /identity/v3/projects => generated 372 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:37:08.638475 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda868e-d265-4ed4-8599-fecbcff9d7be admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:08.648973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:37:08.657829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda868e-d265-4ed4-8599-fecbcff9d7be admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:08.738397 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50752365-f6a2-4f36-8703-14a2db6e1e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:08.739211 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50752365-f6a2-4f36-8703-14a2db6e1e02 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:08.739640 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50752365-f6a2-4f36-8703-14a2db6e1e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:08.739987 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50752365-f6a2-4f36-8703-14a2db6e1e02 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:08.740679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50752365-f6a2-4f36-8703-14a2db6e1e02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:08.742764 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50752365-f6a2-4f36-8703-14a2db6e1e02 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:08.774005 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1167/2354] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:37:08 2022] POST /identity/v3/users => generated 287 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:37:08.779126 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b08ccf-0e92-4526-adf5-42a02b8bf3db admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:08.794483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b08ccf-0e92-4526-adf5-42a02b8bf3db admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:08.800293 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e0d4c3-eee9-46fa-8495-b656ba488350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:08.800946 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0d4c3-eee9-46fa-8495-b656ba488350 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:08.801249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0d4c3-eee9-46fa-8495-b656ba488350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:08.801505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0d4c3-eee9-46fa-8495-b656ba488350 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:08.802225 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e0d4c3-eee9-46fa-8495-b656ba488350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:08.803937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e0d4c3-eee9-46fa-8495-b656ba488350 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:08.825627 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1188/2355] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:37:08 2022] GET /identity/v3/roles => generated 1427 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:37:08.830602 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50752365-f6a2-4f36-8703-14a2db6e1e02 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:08.844435 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50752365-f6a2-4f36-8703-14a2db6e1e02 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:08.851662 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643d8f1-0b5e-48cf-a1db-830d9b787d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:08.852491 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643d8f1-0b5e-48cf-a1db-830d9b787d9d admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:08.852999 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643d8f1-0b5e-48cf-a1db-830d9b787d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:08.853165 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643d8f1-0b5e-48cf-a1db-830d9b787d9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:08.854067 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7643d8f1-0b5e-48cf-a1db-830d9b787d9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:08.855741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7643d8f1-0b5e-48cf-a1db-830d9b787d9d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:08.869914 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1168/2356] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:37:08 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:37:08.874895 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e0d4c3-eee9-46fa-8495-b656ba488350 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:08.886780 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e0d4c3-eee9-46fa-8495-b656ba488350 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:08.895061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee36453-2156-40bc-bacd-08871c435008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:08.896135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee36453-2156-40bc-bacd-08871c435008 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:08.896135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee36453-2156-40bc-bacd-08871c435008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:08.896530 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee36453-2156-40bc-bacd-08871c435008 admin admin] PATH_INFO: `/v3/projects/bee0d982e3ba4f7dba65f51636856b5a/users/6921033062424e14bc30f9b94bf911db/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:08.917245 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee36453-2156-40bc-bacd-08871c435008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bee0d982e3ba4f7dba65f51636856b5a, user_id=6921033062424e14bc30f9b94bf911db, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:08.920441 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee36453-2156-40bc-bacd-08871c435008 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:08.938114 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1189/2357] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:37:08 2022] PUT /identity/v3/projects/bee0d982e3ba4f7dba65f51636856b5a/users/6921033062424e14bc30f9b94bf911db/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:37:08.944290 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1cb08-d6b0-4028-9deb-3c31589e9f78 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:08.944584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1cb08-d6b0-4028-9deb-3c31589e9f78 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:08.944840 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1cb08-d6b0-4028-9deb-3c31589e9f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:08.967777 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-cfa1cb08-d6b0-4028-9deb-3c31589e9f78 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:37:08.969156 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cfa1cb08-d6b0-4028-9deb-3c31589e9f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:09.041751 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1169/2358] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:37:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:37:09.054345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee36453-2156-40bc-bacd-08871c435008 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:09.057158 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-3ee36453-2156-40bc-bacd-08871c435008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:09.083202 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:37:09.132453 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee36453-2156-40bc-bacd-08871c435008 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:09.136635 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76973f0-40e3-4fb7-8b33-9e41a67b2142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:09.137334 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76973f0-40e3-4fb7-8b33-9e41a67b2142 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:09.137620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76973f0-40e3-4fb7-8b33-9e41a67b2142 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:09.137897 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76973f0-40e3-4fb7-8b33-9e41a67b2142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:09.149883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76973f0-40e3-4fb7-8b33-9e41a67b2142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:09.151477 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76973f0-40e3-4fb7-8b33-9e41a67b2142 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:09.164993 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1190/2359] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:37:09 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:37:14.175311 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36085db4-0a2f-4658-9519-8ca9fa45e649 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:14.176304 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36085db4-0a2f-4658-9519-8ca9fa45e649 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:14.176304 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36085db4-0a2f-4658-9519-8ca9fa45e649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:14.214599 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-36085db4-0a2f-4658-9519-8ca9fa45e649 None None] MFA Rules not processed for user `6921033062424e14bc30f9b94bf911db`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:37:14.216012 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-36085db4-0a2f-4658-9519-8ca9fa45e649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:14.332158 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76973f0-40e3-4fb7-8b33-9e41a67b2142 service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:14.346949 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1170/2360] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:37:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:37:14.364667 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76973f0-40e3-4fb7-8b33-9e41a67b2142 service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:14.382103 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a377e1-73b6-4b81-8069-f335903cefa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:14.382677 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a377e1-73b6-4b81-8069-f335903cefa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:14.383025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a377e1-73b6-4b81-8069-f335903cefa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:14.383242 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a377e1-73b6-4b81-8069-f335903cefa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:14.384039 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a377e1-73b6-4b81-8069-f335903cefa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:14.386215 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a377e1-73b6-4b81-8069-f335903cefa0 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:14.426038 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1191/2361] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:37:14 2022] POST /identity/v3/projects => generated 370 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:37:14.431132 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36085db4-0a2f-4658-9519-8ca9fa45e649 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:14.451424 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36085db4-0a2f-4658-9519-8ca9fa45e649 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:14.465746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:37:14.543541 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed871c-880c-4cbc-8ed3-da790c2c4e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:14.544629 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed871c-880c-4cbc-8ed3-da790c2c4e00 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:14.545241 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed871c-880c-4cbc-8ed3-da790c2c4e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:14.545671 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed871c-880c-4cbc-8ed3-da790c2c4e00 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:14.546810 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ed871c-880c-4cbc-8ed3-da790c2c4e00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:14.549903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ed871c-880c-4cbc-8ed3-da790c2c4e00 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:14.588116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1171/2362] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:37:14 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:37:14.594249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a377e1-73b6-4b81-8069-f335903cefa0 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:14.614631 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a377e1-73b6-4b81-8069-f335903cefa0 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:14.623051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee912e-492c-43e7-9659-95bdc39b59b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:14.623835 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee912e-492c-43e7-9659-95bdc39b59b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:14.624400 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee912e-492c-43e7-9659-95bdc39b59b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:14.624895 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee912e-492c-43e7-9659-95bdc39b59b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:14.626286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee912e-492c-43e7-9659-95bdc39b59b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:14.630317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee912e-492c-43e7-9659-95bdc39b59b0 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:14.651699 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1192/2363] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:37:14 2022] GET /identity/v3/roles => generated 1427 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:37:14.657166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed871c-880c-4cbc-8ed3-da790c2c4e00 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:14.669490 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed871c-880c-4cbc-8ed3-da790c2c4e00 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:14.677446 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa7e0a-a6e4-410f-8542-351500e38289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:14.678562 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa7e0a-a6e4-410f-8542-351500e38289 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:14.679119 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa7e0a-a6e4-410f-8542-351500e38289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:14.679679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa7e0a-a6e4-410f-8542-351500e38289 admin admin] PATH_INFO: `/v3/projects/fc5257a153a444dfbf0c14f378bd88c5/users/3fe3e0b894ae428d9a9a52377902e062/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:14.710744 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa7e0a-a6e4-410f-8542-351500e38289 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc5257a153a444dfbf0c14f378bd88c5, user_id=3fe3e0b894ae428d9a9a52377902e062, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:14.714544 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa7e0a-a6e4-410f-8542-351500e38289 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:14.735659 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1172/2364] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:37:14 2022] PUT /identity/v3/projects/fc5257a153a444dfbf0c14f378bd88c5/users/3fe3e0b894ae428d9a9a52377902e062/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:37:14.741326 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee912e-492c-43e7-9659-95bdc39b59b0 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:14.761406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee912e-492c-43e7-9659-95bdc39b59b0 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:14.849612 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20ff3c6-5a92-4638-baff-c123c8db09da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:14.850813 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ff3c6-5a92-4638-baff-c123c8db09da admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:14.851612 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ff3c6-5a92-4638-baff-c123c8db09da admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:14.852215 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ff3c6-5a92-4638-baff-c123c8db09da admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:14.853435 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20ff3c6-5a92-4638-baff-c123c8db09da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:14.856197 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20ff3c6-5a92-4638-baff-c123c8db09da admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:14.873900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1193/2365] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:37:14 2022] GET /identity/v3/roles => generated 1427 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:37:14.880673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa7e0a-a6e4-410f-8542-351500e38289 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:14.893882 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa7e0a-a6e4-410f-8542-351500e38289 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:14.899227 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4851a7c-7ad0-4d7f-85b3-6ae2494e8320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:14.900144 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4851a7c-7ad0-4d7f-85b3-6ae2494e8320 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:14.900251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4851a7c-7ad0-4d7f-85b3-6ae2494e8320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:14.900533 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4851a7c-7ad0-4d7f-85b3-6ae2494e8320 admin admin] PATH_INFO: `/v3/projects/fc5257a153a444dfbf0c14f378bd88c5/users/3fe3e0b894ae428d9a9a52377902e062/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:14.916068 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4851a7c-7ad0-4d7f-85b3-6ae2494e8320 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc5257a153a444dfbf0c14f378bd88c5, user_id=3fe3e0b894ae428d9a9a52377902e062, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:14.923169 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4851a7c-7ad0-4d7f-85b3-6ae2494e8320 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:14.945653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1173/2366] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:37:14 2022] PUT /identity/v3/projects/fc5257a153a444dfbf0c14f378bd88c5/users/3fe3e0b894ae428d9a9a52377902e062/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:37:20.347808 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e9515-625b-4500-8d88-100aff3ab656 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:20.348457 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e9515-625b-4500-8d88-100aff3ab656 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:20.348457 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e9515-625b-4500-8d88-100aff3ab656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:20.387354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-197e9515-625b-4500-8d88-100aff3ab656 None None] MFA Rules not processed for user `3fe3e0b894ae428d9a9a52377902e062`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:37:20.389236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-197e9515-625b-4500-8d88-100aff3ab656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:20.528488 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1194/2367] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:37:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:37:20.551167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4851a7c-7ad0-4d7f-85b3-6ae2494e8320 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:20.553194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a4851a7c-7ad0-4d7f-85b3-6ae2494e8320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:20.663301 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4851a7c-7ad0-4d7f-85b3-6ae2494e8320 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:20.675346 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d58a5d-1e4b-450d-b224-0ef6e1c2a6c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:20.676208 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d58a5d-1e4b-450d-b224-0ef6e1c2a6c8 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:20.676500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d58a5d-1e4b-450d-b224-0ef6e1c2a6c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:20.676779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d58a5d-1e4b-450d-b224-0ef6e1c2a6c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:20.747287 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d58a5d-1e4b-450d-b224-0ef6e1c2a6c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:20.749001 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d58a5d-1e4b-450d-b224-0ef6e1c2a6c8 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:20.755601 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:37:20.761262 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1174/2368] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:37:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:37:22.089461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d94f9-7350-446d-9dde-7e47a854ba67 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:22.089996 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d94f9-7350-446d-9dde-7e47a854ba67 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:22.089996 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d94f9-7350-446d-9dde-7e47a854ba67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:22.114512 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-077d94f9-7350-446d-9dde-7e47a854ba67 None None] MFA Rules not processed for user `6921033062424e14bc30f9b94bf911db`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:37:22.115881 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-077d94f9-7350-446d-9dde-7e47a854ba67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:22.124699 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1195/2369] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:37:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:37:32.106814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:37:39.857188 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d58a5d-1e4b-450d-b224-0ef6e1c2a6c8 service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:39.859032 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-62d58a5d-1e4b-450d-b224-0ef6e1c2a6c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:39.952666 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d58a5d-1e4b-450d-b224-0ef6e1c2a6c8 service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:39.957625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7d5b0-a1f0-4b31-aed4-881d0dfd8054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:39.958401 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7d5b0-a1f0-4b31-aed4-881d0dfd8054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:39.958560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7d5b0-a1f0-4b31-aed4-881d0dfd8054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:39.958842 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7d5b0-a1f0-4b31-aed4-881d0dfd8054 admin admin] PATH_INFO: `/v3/users/30992f46b39740ef86af650859fb48d5` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:39.960091 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7d5b0-a1f0-4b31-aed4-881d0dfd8054 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30992f46b39740ef86af650859fb48d5)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:39.962078 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7d5b0-a1f0-4b31-aed4-881d0dfd8054 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:40.028971 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c7b7d5b0-a1f0-4b31-aed4-881d0dfd8054 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30992f46b39740ef86af650859fb48d5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:40.039064 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c7b7d5b0-a1f0-4b31-aed4-881d0dfd8054 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30992f46b39740ef86af650859fb48d5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:40.039894 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c7b7d5b0-a1f0-4b31-aed4-881d0dfd8054 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30992f46b39740ef86af650859fb48d5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:40.064247 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c7b7d5b0-a1f0-4b31-aed4-881d0dfd8054 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30992f46b39740ef86af650859fb48d5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:40.065006 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c7b7d5b0-a1f0-4b31-aed4-881d0dfd8054 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30992f46b39740ef86af650859fb48d5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:40.091817 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1175/2370] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:37:39 2022] DELETE /identity/v3/users/30992f46b39740ef86af650859fb48d5 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:37:40.097269 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd90e44-7bf7-4c3d-aae3-6b5d54a7cdc1 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:40.097982 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd90e44-7bf7-4c3d-aae3-6b5d54a7cdc1 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:40.097982 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd90e44-7bf7-4c3d-aae3-6b5d54a7cdc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:40.119724 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-1fd90e44-7bf7-4c3d-aae3-6b5d54a7cdc1 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:37:40.121054 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-1fd90e44-7bf7-4c3d-aae3-6b5d54a7cdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:40.197312 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1196/2371] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:37:40 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:37:40.209789 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7d5b0-a1f0-4b31-aed4-881d0dfd8054 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:40.211477 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c7b7d5b0-a1f0-4b31-aed4-881d0dfd8054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:40.293615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7d5b0-a1f0-4b31-aed4-881d0dfd8054 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:40.300403 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497f89a5-07b3-4e3b-afbe-2806cdcd38d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:40.301599 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f89a5-07b3-4e3b-afbe-2806cdcd38d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:40.302091 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f89a5-07b3-4e3b-afbe-2806cdcd38d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:40.302570 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f89a5-07b3-4e3b-afbe-2806cdcd38d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:40.319289 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497f89a5-07b3-4e3b-afbe-2806cdcd38d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:40.321530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497f89a5-07b3-4e3b-afbe-2806cdcd38d0 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:40.338662 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1176/2372] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:37:40 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:37:40.655732 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd90e44-7bf7-4c3d-aae3-6b5d54a7cdc1 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:40.657476 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-1fd90e44-7bf7-4c3d-aae3-6b5d54a7cdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:40.675395 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd90e44-7bf7-4c3d-aae3-6b5d54a7cdc1 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:40.685793 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58180a2-e269-4d56-879e-747b5b8745e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:40.686835 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58180a2-e269-4d56-879e-747b5b8745e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:40.687327 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58180a2-e269-4d56-879e-747b5b8745e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:40.687814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58180a2-e269-4d56-879e-747b5b8745e4 admin admin] PATH_INFO: `/v3/projects/39068c2e75774c41ab345bba771c00ea` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:40.689834 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58180a2-e269-4d56-879e-747b5b8745e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39068c2e75774c41ab345bba771c00ea)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:40.692708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58180a2-e269-4d56-879e-747b5b8745e4 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:40.761381 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a58180a2-e269-4d56-879e-747b5b8745e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39068c2e75774c41ab345bba771c00ea', 'request_id': 'req-a58180a2-e269-4d56-879e-747b5b8745e4'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:40.771667 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a58180a2-e269-4d56-879e-747b5b8745e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39068c2e75774c41ab345bba771c00ea', 'request_id': 'req-a58180a2-e269-4d56-879e-747b5b8745e4'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:40.788750 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a58180a2-e269-4d56-879e-747b5b8745e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39068c2e75774c41ab345bba771c00ea', 'request_id': 'req-a58180a2-e269-4d56-879e-747b5b8745e4'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:40.805853 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a58180a2-e269-4d56-879e-747b5b8745e4 admin admin] The token cache is being invalidate because project 39068c2e75774c41ab345bba771c00ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:37:40.806151 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a58180a2-e269-4d56-879e-747b5b8745e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:40.806972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a58180a2-e269-4d56-879e-747b5b8745e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:37:40.808873 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1197/2373] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:37:40 2022] DELETE /identity/v3/projects/39068c2e75774c41ab345bba771c00ea => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:37:43.605597 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497f89a5-07b3-4e3b-afbe-2806cdcd38d0 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:43.617006 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497f89a5-07b3-4e3b-afbe-2806cdcd38d0 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:43.652887 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:37:43.693079 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b300d731-215b-4180-8c27-edfe86fe4b5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:43.693721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300d731-215b-4180-8c27-edfe86fe4b5e service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:43.694028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300d731-215b-4180-8c27-edfe86fe4b5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:43.694336 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300d731-215b-4180-8c27-edfe86fe4b5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:43.696306 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b300d731-215b-4180-8c27-edfe86fe4b5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:43.792108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b300d731-215b-4180-8c27-edfe86fe4b5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:43.794527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b300d731-215b-4180-8c27-edfe86fe4b5e service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:43.807699 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1177/2374] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:37:43 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:37:45.025759 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58180a2-e269-4d56-879e-747b5b8745e4 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:45.029302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-a58180a2-e269-4d56-879e-747b5b8745e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:45.049468 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58180a2-e269-4d56-879e-747b5b8745e4 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:45.055254 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53dd02d-b959-4977-9fed-e10ea9e15827 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:45.055604 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53dd02d-b959-4977-9fed-e10ea9e15827 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:45.055887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53dd02d-b959-4977-9fed-e10ea9e15827 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:45.056198 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53dd02d-b959-4977-9fed-e10ea9e15827 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:45.058324 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c53dd02d-b959-4977-9fed-e10ea9e15827 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:45.080707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53dd02d-b959-4977-9fed-e10ea9e15827 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:45.082029 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53dd02d-b959-4977-9fed-e10ea9e15827 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:45.096321 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1198/2375] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:37:45 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:37:49.966430 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b300d731-215b-4180-8c27-edfe86fe4b5e service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:49.968133 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b300d731-215b-4180-8c27-edfe86fe4b5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:49.982874 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b300d731-215b-4180-8c27-edfe86fe4b5e service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:49.990251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c6bfd-0b13-4426-9188-b29b071532cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:49.990251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c6bfd-0b13-4426-9188-b29b071532cf service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:49.990251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c6bfd-0b13-4426-9188-b29b071532cf service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:49.990901 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c6bfd-0b13-4426-9188-b29b071532cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:49.992384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ee6c6bfd-0b13-4426-9188-b29b071532cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:50.082054 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c6bfd-0b13-4426-9188-b29b071532cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:50.084435 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c6bfd-0b13-4426-9188-b29b071532cf service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:50.100710 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1178/2376] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:37:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:37:52.773826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae444000-f10e-4a34-a793-23ecd5f8f445 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:52.774539 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae444000-f10e-4a34-a793-23ecd5f8f445 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:52.774539 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae444000-f10e-4a34-a793-23ecd5f8f445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:52.798489 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-ae444000-f10e-4a34-a793-23ecd5f8f445 None None] MFA Rules not processed for user `41a38b5af1654f44bc6cb5832c2e8e87`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:37:52.800167 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-ae444000-f10e-4a34-a793-23ecd5f8f445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:52.872051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1199/2377] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:37:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:37:57.429652 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c6bfd-0b13-4426-9188-b29b071532cf service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:37:57.451609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c6bfd-0b13-4426-9188-b29b071532cf service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:37:57.457795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5973dd4-d041-439a-b386-03febe6f6887 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:37:57.458454 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5973dd4-d041-439a-b386-03febe6f6887 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:37:57.458737 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5973dd4-d041-439a-b386-03febe6f6887 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:37:57.459015 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5973dd4-d041-439a-b386-03febe6f6887 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:37:57.461617 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f5973dd4-d041-439a-b386-03febe6f6887 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:37:57.476186 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5973dd4-d041-439a-b386-03febe6f6887 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:37:57.478061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5973dd4-d041-439a-b386-03febe6f6887 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:37:57.493114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1179/2378] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:37:57 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:38:02.865424 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:38:07.057464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae444000-f10e-4a34-a793-23ecd5f8f445 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:38:07.077102 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae444000-f10e-4a34-a793-23ecd5f8f445 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:38:07.085200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b56e5-e8f5-4686-8b7e-9f4f64623d2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:38:07.086339 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b56e5-e8f5-4686-8b7e-9f4f64623d2a service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:38:07.086896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b56e5-e8f5-4686-8b7e-9f4f64623d2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:38:07.087654 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b56e5-e8f5-4686-8b7e-9f4f64623d2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:38:07.091503 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-e64b56e5-e8f5-4686-8b7e-9f4f64623d2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:38:07.110462 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b56e5-e8f5-4686-8b7e-9f4f64623d2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:38:07.113524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b56e5-e8f5-4686-8b7e-9f4f64623d2a service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:38:07.129102 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1200/2379] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:38:07 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:38:07.486168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=78 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:38:07.817158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5973dd4-d041-439a-b386-03febe6f6887 service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:38:07.818943 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f5973dd4-d041-439a-b386-03febe6f6887 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:38:07.918170 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5973dd4-d041-439a-b386-03febe6f6887 service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:38:07.924885 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480c8de-0143-4aba-9998-82a5fd76c6d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c7cfbe911c4378857848966f4f8d98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:38:07.925650 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480c8de-0143-4aba-9998-82a5fd76c6d4 service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:38:07.925928 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480c8de-0143-4aba-9998-82a5fd76c6d4 service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:38:07.926209 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480c8de-0143-4aba-9998-82a5fd76c6d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:38:07.928942 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c480c8de-0143-4aba-9998-82a5fd76c6d4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:38:07.944754 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480c8de-0143-4aba-9998-82a5fd76c6d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:38:07.946542 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480c8de-0143-4aba-9998-82a5fd76c6d4 service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:38:07.958828 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1180/2380] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:38:07 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:38:17.122941 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=6 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:38:17.953573 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=32 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:38:27.722822 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b56e5-e8f5-4686-8b7e-9f4f64623d2a service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:38:27.735549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b56e5-e8f5-4686-8b7e-9f4f64623d2a service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:38:27.741843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef60dcb-0023-47fd-a84b-0ffb4b8dbfea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:38:27.742761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef60dcb-0023-47fd-a84b-0ffb4b8dbfea service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:38:27.743090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef60dcb-0023-47fd-a84b-0ffb4b8dbfea service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:38:27.743463 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef60dcb-0023-47fd-a84b-0ffb4b8dbfea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:38:27.746333 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-1ef60dcb-0023-47fd-a84b-0ffb4b8dbfea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:38:27.758213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef60dcb-0023-47fd-a84b-0ffb4b8dbfea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:38:27.759723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef60dcb-0023-47fd-a84b-0ffb4b8dbfea service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:38:27.768946 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1201/2381] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:38:27 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:38:37.765039 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=6 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:38:41.947530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685765b8-aa3a-4059-8f8b-a8f8a36db2dc None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:38:41.948231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685765b8-aa3a-4059-8f8b-a8f8a36db2dc None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:38:41.948231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685765b8-aa3a-4059-8f8b-a8f8a36db2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:38:41.970883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-685765b8-aa3a-4059-8f8b-a8f8a36db2dc None None] MFA Rules not processed for user `db9e00718840481ca416e25ef436bb24`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:38:41.972229 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-685765b8-aa3a-4059-8f8b-a8f8a36db2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:38:42.041228 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1181/2382] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:38:41 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:38:46.263676 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef60dcb-0023-47fd-a84b-0ffb4b8dbfea service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:38:46.265302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-1ef60dcb-0023-47fd-a84b-0ffb4b8dbfea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:38:46.283120 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef60dcb-0023-47fd-a84b-0ffb4b8dbfea service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:38:46.290377 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2cf95-efde-4132-9020-868d43c0998c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:38:46.291228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2cf95-efde-4132-9020-868d43c0998c service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:38:46.291518 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2cf95-efde-4132-9020-868d43c0998c service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:38:46.291799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2cf95-efde-4132-9020-868d43c0998c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:38:46.294094 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-18a2cf95-efde-4132-9020-868d43c0998c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:38:46.309157 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2cf95-efde-4132-9020-868d43c0998c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:38:46.310931 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2cf95-efde-4132-9020-868d43c0998c service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:38:46.327274 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1202/2383] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:38:46 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:38:48.016417 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685765b8-aa3a-4059-8f8b-a8f8a36db2dc None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:38:48.027728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685765b8-aa3a-4059-8f8b-a8f8a36db2dc None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:38:48.031975 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5b1c4-1ac0-4798-8195-c78ab35a9bd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:38:48.032641 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5b1c4-1ac0-4798-8195-c78ab35a9bd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:38:48.032928 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5b1c4-1ac0-4798-8195-c78ab35a9bd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:38:48.033209 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5b1c4-1ac0-4798-8195-c78ab35a9bd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:38:48.035232 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d0c5b1c4-1ac0-4798-8195-c78ab35a9bd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:38:48.114502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c5b1c4-1ac0-4798-8195-c78ab35a9bd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:38:48.116137 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c5b1c4-1ac0-4798-8195-c78ab35a9bd0 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:38:48.126615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1182/2384] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:38:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:38:49.904737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2cf95-efde-4132-9020-868d43c0998c service nova] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:38:49.917480 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2cf95-efde-4132-9020-868d43c0998c service nova] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:38:49.929315 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db666d7f-fb61-4701-b6bd-559f02d1908d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:38:49.931321 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db666d7f-fb61-4701-b6bd-559f02d1908d service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:38:49.931861 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db666d7f-fb61-4701-b6bd-559f02d1908d service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:38:49.932503 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db666d7f-fb61-4701-b6bd-559f02d1908d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:38:49.934924 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-db666d7f-fb61-4701-b6bd-559f02d1908d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:38:49.952296 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db666d7f-fb61-4701-b6bd-559f02d1908d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:38:49.953389 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db666d7f-fb61-4701-b6bd-559f02d1908d service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:38:49.967873 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1203/2385] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:38:49 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:38:55.414670 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5b1c4-1ac0-4798-8195-c78ab35a9bd0 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:38:55.429884 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5b1c4-1ac0-4798-8195-c78ab35a9bd0 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:38:55.439667 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23301063-b877-4c7d-b8d6-31cc36aa5301 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:38:55.439667 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23301063-b877-4c7d-b8d6-31cc36aa5301 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:38:55.439667 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23301063-b877-4c7d-b8d6-31cc36aa5301 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:38:55.440228 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23301063-b877-4c7d-b8d6-31cc36aa5301 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:38:55.450112 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-23301063-b877-4c7d-b8d6-31cc36aa5301 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:38:55.474415 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23301063-b877-4c7d-b8d6-31cc36aa5301 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:38:55.478182 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23301063-b877-4c7d-b8d6-31cc36aa5301 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:38:55.497932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1183/2386] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:38:55 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:38:59.962391 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=12 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:39:05.491263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=61 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:39:59.356455 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d682bab-9889-4dd4-beb5-3e777188270c None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:39:59.357255 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d682bab-9889-4dd4-beb5-3e777188270c None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:39:59.357255 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d682bab-9889-4dd4-beb5-3e777188270c None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:39:59.358497 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1204/2387] 10.222.0.121 () {54 vars in 926 bytes} [Thu Aug 4 10:39:59 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:39:59.368438 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76aaea-6c5c-4bd1-b546-56ac77479fd9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:39:59.368972 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76aaea-6c5c-4bd1-b546-56ac77479fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:39:59.368972 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76aaea-6c5c-4bd1-b546-56ac77479fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:39:59.395122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-cf76aaea-6c5c-4bd1-b546-56ac77479fd9 None None] MFA Rules not processed for user `01c7cfbe911c4378857848966f4f8d98`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:39:59.396599 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cf76aaea-6c5c-4bd1-b546-56ac77479fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:39:59.405701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1184/2388] 10.222.0.121 () {58 vars in 1023 bytes} [Thu Aug 4 10:39:59 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:39:59.418443 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d682bab-9889-4dd4-beb5-3e777188270c None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:39:59.433952 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d682bab-9889-4dd4-beb5-3e777188270c None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:39:59.439613 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99dedb6-805b-474c-b443-e5f00acaa493 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c7cfbe911c4378857848966f4f8d98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:39:59.440381 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dedb6-805b-474c-b443-e5f00acaa493 service placement] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:39:59.440732 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dedb6-805b-474c-b443-e5f00acaa493 service placement] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:39:59.441114 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dedb6-805b-474c-b443-e5f00acaa493 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:39:59.451003 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99dedb6-805b-474c-b443-e5f00acaa493 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:39:59.453370 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99dedb6-805b-474c-b443-e5f00acaa493 service placement] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:39:59.455562 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76aaea-6c5c-4bd1-b546-56ac77479fd9 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:39:59.457911 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cf76aaea-6c5c-4bd1-b546-56ac77479fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:39:59.467472 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1205/2389] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:39:59 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:39:59.543850 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76aaea-6c5c-4bd1-b546-56ac77479fd9 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:39:59.550012 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d205dfb-0407-4917-9c69-ff94a156d453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:39:59.550901 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d205dfb-0407-4917-9c69-ff94a156d453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:39:59.551090 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d205dfb-0407-4917-9c69-ff94a156d453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:39:59.551451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d205dfb-0407-4917-9c69-ff94a156d453 admin admin] PATH_INFO: `/v3/users/db9e00718840481ca416e25ef436bb24` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:39:59.552644 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d205dfb-0407-4917-9c69-ff94a156d453 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db9e00718840481ca416e25ef436bb24)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:39:59.554820 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d205dfb-0407-4917-9c69-ff94a156d453 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:39:59.626499 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1d205dfb-0407-4917-9c69-ff94a156d453 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db9e00718840481ca416e25ef436bb24'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:39:59.637204 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1d205dfb-0407-4917-9c69-ff94a156d453 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db9e00718840481ca416e25ef436bb24'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:39:59.638171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1d205dfb-0407-4917-9c69-ff94a156d453 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db9e00718840481ca416e25ef436bb24'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:39:59.670279 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1d205dfb-0407-4917-9c69-ff94a156d453 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db9e00718840481ca416e25ef436bb24'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:39:59.671195 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1d205dfb-0407-4917-9c69-ff94a156d453 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db9e00718840481ca416e25ef436bb24'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:39:59.690303 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1185/2390] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:39:59 2022] DELETE /identity/v3/users/db9e00718840481ca416e25ef436bb24 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:39:59.919279 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99dedb6-805b-474c-b443-e5f00acaa493 service placement] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:39:59.932629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99dedb6-805b-474c-b443-e5f00acaa493 service placement] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:39:59.994212 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f6cdb-47d4-47b7-be94-0d724a45d378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:39:59.994879 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f6cdb-47d4-47b7-be94-0d724a45d378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:39:59.995168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f6cdb-47d4-47b7-be94-0d724a45d378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:39:59.995446 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f6cdb-47d4-47b7-be94-0d724a45d378 admin admin] PATH_INFO: `/v3/projects/aacec5c4a2584f60af1ca9843e9e8732` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:39:59.996738 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f6cdb-47d4-47b7-be94-0d724a45d378 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aacec5c4a2584f60af1ca9843e9e8732)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:39:59.998683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f6cdb-47d4-47b7-be94-0d724a45d378 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:40:00.055045 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7a5f6cdb-47d4-47b7-be94-0d724a45d378 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aacec5c4a2584f60af1ca9843e9e8732', 'request_id': 'req-7a5f6cdb-47d4-47b7-be94-0d724a45d378'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:40:00.064596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7a5f6cdb-47d4-47b7-be94-0d724a45d378 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aacec5c4a2584f60af1ca9843e9e8732', 'request_id': 'req-7a5f6cdb-47d4-47b7-be94-0d724a45d378'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:40:00.071857 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7a5f6cdb-47d4-47b7-be94-0d724a45d378 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aacec5c4a2584f60af1ca9843e9e8732', 'request_id': 'req-7a5f6cdb-47d4-47b7-be94-0d724a45d378'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:40:00.085492 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7a5f6cdb-47d4-47b7-be94-0d724a45d378 admin admin] The token cache is being invalidate because project aacec5c4a2584f60af1ca9843e9e8732 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:40:00.085790 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7a5f6cdb-47d4-47b7-be94-0d724a45d378 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:40:00.086555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7a5f6cdb-47d4-47b7-be94-0d724a45d378 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:40:00.088303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1206/2391] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:39:59 2022] DELETE /identity/v3/projects/aacec5c4a2584f60af1ca9843e9e8732 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:40:05.017230 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d205dfb-0407-4917-9c69-ff94a156d453 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:40:05.019273 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1d205dfb-0407-4917-9c69-ff94a156d453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:40:05.111290 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d205dfb-0407-4917-9c69-ff94a156d453 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:40:05.120642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:40:05.186928 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a0baf8-9e08-4146-8ae9-8651a19cb01e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:40:05.187602 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0baf8-9e08-4146-8ae9-8651a19cb01e service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:40:05.187883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0baf8-9e08-4146-8ae9-8651a19cb01e service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:40:05.188264 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0baf8-9e08-4146-8ae9-8651a19cb01e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:40:05.190225 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-60a0baf8-9e08-4146-8ae9-8651a19cb01e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:40:05.257302 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0baf8-9e08-4146-8ae9-8651a19cb01e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:40:05.259141 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0baf8-9e08-4146-8ae9-8651a19cb01e service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:40:05.271116 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1186/2392] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:40:05 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:40:10.081995 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:40:15.266137 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=51 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:40:29.298606 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c687c4-7ff3-42ee-9355-aa07b1935c94 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:40:29.299334 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c687c4-7ff3-42ee-9355-aa07b1935c94 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:40:29.299334 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c687c4-7ff3-42ee-9355-aa07b1935c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:40:29.320935 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-c3c687c4-7ff3-42ee-9355-aa07b1935c94 None None] MFA Rules not processed for user `3170dc3323f54512bc8819c8efa86b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:40:29.322247 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c3c687c4-7ff3-42ee-9355-aa07b1935c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:40:29.332031 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1207/2393] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:40:29 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:40:39.314768 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:40:55.645168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580847db-01d1-46fa-bb65-e02e77c85bf3 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:40:55.645788 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580847db-01d1-46fa-bb65-e02e77c85bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:40:55.645788 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580847db-01d1-46fa-bb65-e02e77c85bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:40:55.676441 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-580847db-01d1-46fa-bb65-e02e77c85bf3 None None] MFA Rules not processed for user `41a38b5af1654f44bc6cb5832c2e8e87`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:40:55.677726 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-580847db-01d1-46fa-bb65-e02e77c85bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:40:55.760803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1187/2394] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:40:55 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:41:05.757655 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=37 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:41:53.231019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c687c4-7ff3-42ee-9355-aa07b1935c94 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:41:53.235076 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c3c687c4-7ff3-42ee-9355-aa07b1935c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:41:53.411199 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c687c4-7ff3-42ee-9355-aa07b1935c94 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:41:53.416833 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e384595-75e7-4911-a735-441460f012c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c7cfbe911c4378857848966f4f8d98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:41:53.417492 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e384595-75e7-4911-a735-441460f012c2 service placement] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:41:53.417775 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e384595-75e7-4911-a735-441460f012c2 service placement] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:41:53.418055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e384595-75e7-4911-a735-441460f012c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:41:53.419894 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-4e384595-75e7-4911-a735-441460f012c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:41:53.435325 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e384595-75e7-4911-a735-441460f012c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:41:53.437224 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e384595-75e7-4911-a735-441460f012c2 service placement] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:41:53.448311 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1208/2395] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:41:53 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:42:01.288996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580847db-01d1-46fa-bb65-e02e77c85bf3 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:42:01.291492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-580847db-01d1-46fa-bb65-e02e77c85bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:42:01.479020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580847db-01d1-46fa-bb65-e02e77c85bf3 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:42:01.486292 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224c1af-d20f-4ddd-92fc-c735e8a3e4f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:42:01.488740 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224c1af-d20f-4ddd-92fc-c735e8a3e4f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:01.489273 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224c1af-d20f-4ddd-92fc-c735e8a3e4f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:01.489949 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224c1af-d20f-4ddd-92fc-c735e8a3e4f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:01.492133 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f224c1af-d20f-4ddd-92fc-c735e8a3e4f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:42:01.506339 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f224c1af-d20f-4ddd-92fc-c735e8a3e4f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:42:01.507936 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f224c1af-d20f-4ddd-92fc-c735e8a3e4f7 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:42:01.522173 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1188/2396] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:42:01 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:42:01.704614 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4a60a-1825-4e2f-bb5f-880468c3eaa8 None None] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:01.705269 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4a60a-1825-4e2f-bb5f-880468c3eaa8 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:01.705269 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4a60a-1825-4e2f-bb5f-880468c3eaa8 None None] PATH_INFO: `/` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:01.707713 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1209/2397] 10.222.0.121 () {54 vars in 926 bytes} [Thu Aug 4 10:42:01 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:42:01.717563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b90566-22fb-4bb1-b964-dac6012d75a9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:01.718004 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b90566-22fb-4bb1-b964-dac6012d75a9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:01.718362 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b90566-22fb-4bb1-b964-dac6012d75a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:01.786332 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b9b90566-22fb-4bb1-b964-dac6012d75a9 None None] MFA Rules not processed for user `90c7eaf4500f4ef496ba72405522006d`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:42:01.788235 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b9b90566-22fb-4bb1-b964-dac6012d75a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:42:01.811392 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1189/2398] 10.222.0.121 () {58 vars in 1023 bytes} [Thu Aug 4 10:42:01 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:42:01.819564 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4a60a-1825-4e2f-bb5f-880468c3eaa8 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:42:01.822961 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d1f4a60a-1825-4e2f-bb5f-880468c3eaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:42:01.845573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4a60a-1825-4e2f-bb5f-880468c3eaa8 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:42:01.850023 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa852e3-65c4-45be-8b17-9a704f33a161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:42:01.850727 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa852e3-65c4-45be-8b17-9a704f33a161 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:01.851203 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa852e3-65c4-45be-8b17-9a704f33a161 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:01.851689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa852e3-65c4-45be-8b17-9a704f33a161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:01.861214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa852e3-65c4-45be-8b17-9a704f33a161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:42:01.863341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa852e3-65c4-45be-8b17-9a704f33a161 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:42:01.886318 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1210/2399] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:42:01 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:42:01.993517 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672dda0-8c36-46ae-a171-7486a42873ed None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:01.994100 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672dda0-8c36-46ae-a171-7486a42873ed None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:01.994384 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672dda0-8c36-46ae-a171-7486a42873ed None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:01.997072 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1190/2400] 10.222.0.121 () {54 vars in 926 bytes} [Thu Aug 4 10:42:01 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:42:02.007072 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4055467-7426-4dc8-8c61-440832ef92a3 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:02.007361 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4055467-7426-4dc8-8c61-440832ef92a3 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:02.007730 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4055467-7426-4dc8-8c61-440832ef92a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:02.072116 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-f4055467-7426-4dc8-8c61-440832ef92a3 None None] MFA Rules not processed for user `0d947aeb749546309a8d4b7938087ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:42:02.074011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-f4055467-7426-4dc8-8c61-440832ef92a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:42:02.147572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1211/2401] 10.222.0.121 () {58 vars in 1023 bytes} [Thu Aug 4 10:42:02 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:42:02.170743 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672dda0-8c36-46ae-a171-7486a42873ed None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:42:02.193152 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672dda0-8c36-46ae-a171-7486a42873ed None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:42:02.202284 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf537e-9d5b-4559-92bd-482fb3c85df1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:42:02.204585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf537e-9d5b-4559-92bd-482fb3c85df1 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:02.205145 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf537e-9d5b-4559-92bd-482fb3c85df1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:02.208588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf537e-9d5b-4559-92bd-482fb3c85df1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:02.234518 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf537e-9d5b-4559-92bd-482fb3c85df1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:42:02.236134 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf537e-9d5b-4559-92bd-482fb3c85df1 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:42:02.251113 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1191/2402] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:42:02 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:42:05.803777 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2542a77-2758-4c63-93fd-c17f0002573e None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:05.804459 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2542a77-2758-4c63-93fd-c17f0002573e None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:05.804459 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2542a77-2758-4c63-93fd-c17f0002573e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:05.828185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-e2542a77-2758-4c63-93fd-c17f0002573e None None] MFA Rules not processed for user `3aa2e2a5a6ed4c5c9db91269266dd30b`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:42:05.829538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-e2542a77-2758-4c63-93fd-c17f0002573e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:42:05.868329 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:42:05.897546 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1212/2403] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:42:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:42:12.244378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=52 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:42:15.890331 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=9 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:42:19.079401 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf537e-9d5b-4559-92bd-482fb3c85df1 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:42:19.094360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf537e-9d5b-4559-92bd-482fb3c85df1 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:42:19.100721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662fa3d0-e2e0-4eb4-b789-9620d71b179c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:42:19.101720 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fa3d0-e2e0-4eb4-b789-9620d71b179c service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:19.102215 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fa3d0-e2e0-4eb4-b789-9620d71b179c service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:19.102690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fa3d0-e2e0-4eb4-b789-9620d71b179c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:19.105615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-662fa3d0-e2e0-4eb4-b789-9620d71b179c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:42:19.210294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662fa3d0-e2e0-4eb4-b789-9620d71b179c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:42:19.211936 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662fa3d0-e2e0-4eb4-b789-9620d71b179c service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:42:19.234747 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1192/2404] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:42:19 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:42:20.386377 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2542a77-2758-4c63-93fd-c17f0002573e None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:42:20.398273 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2542a77-2758-4c63-93fd-c17f0002573e None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:42:20.402972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70b36f-7669-42a1-84e6-5b206fb96586 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:42:20.403842 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70b36f-7669-42a1-84e6-5b206fb96586 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:20.404340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70b36f-7669-42a1-84e6-5b206fb96586 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:20.404881 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70b36f-7669-42a1-84e6-5b206fb96586 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:20.407281 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-ad70b36f-7669-42a1-84e6-5b206fb96586 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:42:20.494582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad70b36f-7669-42a1-84e6-5b206fb96586 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:42:20.497116 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad70b36f-7669-42a1-84e6-5b206fb96586 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:42:20.511562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1213/2405] 10.222.0.121 () {60 vars in 1401 bytes} [Thu Aug 4 10:42:20 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3258 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:42:23.715582 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662fa3d0-e2e0-4eb4-b789-9620d71b179c service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:42:23.717179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-662fa3d0-e2e0-4eb4-b789-9620d71b179c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:42:23.730879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662fa3d0-e2e0-4eb4-b789-9620d71b179c service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:42:23.736658 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a2c4e2-5dd7-44b3-95da-c0d16aaeebe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:42:23.737342 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2c4e2-5dd7-44b3-95da-c0d16aaeebe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:23.737627 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2c4e2-5dd7-44b3-95da-c0d16aaeebe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:23.737924 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2c4e2-5dd7-44b3-95da-c0d16aaeebe0 admin admin] PATH_INFO: `/v3/users/3170dc3323f54512bc8819c8efa86b0f` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:23.739197 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a2c4e2-5dd7-44b3-95da-c0d16aaeebe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3170dc3323f54512bc8819c8efa86b0f)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:42:23.741226 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a2c4e2-5dd7-44b3-95da-c0d16aaeebe0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:42:23.817109 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a1a2c4e2-5dd7-44b3-95da-c0d16aaeebe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3170dc3323f54512bc8819c8efa86b0f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:42:23.825475 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a1a2c4e2-5dd7-44b3-95da-c0d16aaeebe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3170dc3323f54512bc8819c8efa86b0f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:42:23.826337 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a1a2c4e2-5dd7-44b3-95da-c0d16aaeebe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3170dc3323f54512bc8819c8efa86b0f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:42:23.841760 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a1a2c4e2-5dd7-44b3-95da-c0d16aaeebe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3170dc3323f54512bc8819c8efa86b0f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:42:23.842502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a1a2c4e2-5dd7-44b3-95da-c0d16aaeebe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3170dc3323f54512bc8819c8efa86b0f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:42:23.860436 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1193/2406] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:42:23 2022] DELETE /identity/v3/users/3170dc3323f54512bc8819c8efa86b0f => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:42:24.095403 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70b36f-7669-42a1-84e6-5b206fb96586 service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:42:24.111816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70b36f-7669-42a1-84e6-5b206fb96586 service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:42:24.200315 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0677d-de6e-4485-b635-2b14efe71009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:42:24.201171 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0677d-de6e-4485-b635-2b14efe71009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:24.201551 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0677d-de6e-4485-b635-2b14efe71009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:24.202051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0677d-de6e-4485-b635-2b14efe71009 admin admin] PATH_INFO: `/v3/projects/7121bd118a2549c39332cfdb7edaa92f` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:24.203841 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef0677d-de6e-4485-b635-2b14efe71009 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7121bd118a2549c39332cfdb7edaa92f)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:42:24.206815 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef0677d-de6e-4485-b635-2b14efe71009 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:42:24.269059 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7ef0677d-de6e-4485-b635-2b14efe71009 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7121bd118a2549c39332cfdb7edaa92f', 'request_id': 'req-7ef0677d-de6e-4485-b635-2b14efe71009'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:42:24.280821 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7ef0677d-de6e-4485-b635-2b14efe71009 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7121bd118a2549c39332cfdb7edaa92f', 'request_id': 'req-7ef0677d-de6e-4485-b635-2b14efe71009'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:42:24.291329 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7ef0677d-de6e-4485-b635-2b14efe71009 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7121bd118a2549c39332cfdb7edaa92f', 'request_id': 'req-7ef0677d-de6e-4485-b635-2b14efe71009'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:42:24.306745 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7ef0677d-de6e-4485-b635-2b14efe71009 admin admin] The token cache is being invalidate because project 7121bd118a2549c39332cfdb7edaa92f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:42:24.307209 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7ef0677d-de6e-4485-b635-2b14efe71009 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:42:24.308160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7ef0677d-de6e-4485-b635-2b14efe71009 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:42:24.310012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1214/2407] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:42:24 2022] DELETE /identity/v3/projects/7121bd118a2549c39332cfdb7edaa92f => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:42:24.316257 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5166ca-0b64-4a25-a0d5-a3585c0ded18 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:24.316694 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5166ca-0b64-4a25-a0d5-a3585c0ded18 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:24.317069 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5166ca-0b64-4a25-a0d5-a3585c0ded18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:24.382869 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-fe5166ca-0b64-4a25-a0d5-a3585c0ded18 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:42:24.384220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fe5166ca-0b64-4a25-a0d5-a3585c0ded18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:42:24.397863 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:42:24.451866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1194/2408] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:42:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:42:24.460149 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a08be1-23c8-4e7f-a851-7813436e9420 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:24.460477 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a08be1-23c8-4e7f-a851-7813436e9420 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:24.460803 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a08be1-23c8-4e7f-a851-7813436e9420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:24.481952 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-c1a08be1-23c8-4e7f-a851-7813436e9420 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:42:24.483226 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c1a08be1-23c8-4e7f-a851-7813436e9420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:42:24.491911 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1215/2409] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:42:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:42:24.496749 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5166ca-0b64-4a25-a0d5-a3585c0ded18 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:42:24.507832 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5166ca-0b64-4a25-a0d5-a3585c0ded18 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:42:24.513215 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9877e-6cdf-494a-ac7b-9be12aab8e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:42:24.513868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9877e-6cdf-494a-ac7b-9be12aab8e88 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:24.514151 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9877e-6cdf-494a-ac7b-9be12aab8e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:24.514525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9877e-6cdf-494a-ac7b-9be12aab8e88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:24.515151 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9877e-6cdf-494a-ac7b-9be12aab8e88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:42:24.515464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9877e-6cdf-494a-ac7b-9be12aab8e88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:42:24.517137 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9877e-6cdf-494a-ac7b-9be12aab8e88 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:42:24.530321 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1195/2410] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:42:24 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:42:24.535387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a08be1-23c8-4e7f-a851-7813436e9420 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:42:24.545858 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a08be1-23c8-4e7f-a851-7813436e9420 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:42:24.550936 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3a078-74fe-42de-a485-b43663169d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:42:24.551578 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3a078-74fe-42de-a485-b43663169d0e admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:24.551862 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3a078-74fe-42de-a485-b43663169d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:24.553285 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3a078-74fe-42de-a485-b43663169d0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:24.553285 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac3a078-74fe-42de-a485-b43663169d0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:42:24.555293 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac3a078-74fe-42de-a485-b43663169d0e admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:42:24.580378 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:42:24.589813 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1216/2411] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:42:24 2022] POST /identity/v3/projects => generated 390 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:42:24.594203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9877e-6cdf-494a-ac7b-9be12aab8e88 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:42:24.606958 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9877e-6cdf-494a-ac7b-9be12aab8e88 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:42:24.694394 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f503010-a080-4b41-88e1-2879e44691e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:42:24.695322 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f503010-a080-4b41-88e1-2879e44691e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:24.695640 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f503010-a080-4b41-88e1-2879e44691e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:24.695913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f503010-a080-4b41-88e1-2879e44691e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:24.696841 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f503010-a080-4b41-88e1-2879e44691e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:42:24.698817 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f503010-a080-4b41-88e1-2879e44691e8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:42:24.728308 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1196/2412] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:42:24 2022] POST /identity/v3/users => generated 296 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:42:24.732797 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3a078-74fe-42de-a485-b43663169d0e admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:42:24.746599 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3a078-74fe-42de-a485-b43663169d0e admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:42:24.755473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f34ac2-a69d-46aa-8672-f00374bfd777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:42:24.756211 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f34ac2-a69d-46aa-8672-f00374bfd777 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:24.756495 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f34ac2-a69d-46aa-8672-f00374bfd777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:24.756775 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f34ac2-a69d-46aa-8672-f00374bfd777 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:24.757550 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f34ac2-a69d-46aa-8672-f00374bfd777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:42:24.759247 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f34ac2-a69d-46aa-8672-f00374bfd777 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:42:24.770972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1217/2413] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:42:24 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:42:24.776067 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f503010-a080-4b41-88e1-2879e44691e8 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:42:24.787852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f503010-a080-4b41-88e1-2879e44691e8 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:42:24.793819 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea5b37-b945-4bf3-884b-79005e24435a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:42:24.794578 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea5b37-b945-4bf3-884b-79005e24435a admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:24.795079 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea5b37-b945-4bf3-884b-79005e24435a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:24.795552 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea5b37-b945-4bf3-884b-79005e24435a admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:24.796593 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea5b37-b945-4bf3-884b-79005e24435a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:42:24.798408 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea5b37-b945-4bf3-884b-79005e24435a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:42:24.814287 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1197/2414] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:42:24 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:42:24.825341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f34ac2-a69d-46aa-8672-f00374bfd777 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:42:24.837129 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f34ac2-a69d-46aa-8672-f00374bfd777 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:42:24.842691 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bd0d4-2f0d-4bc5-ad7c-3efeef4e8ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:42:24.843445 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bd0d4-2f0d-4bc5-ad7c-3efeef4e8ceb admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:24.843725 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bd0d4-2f0d-4bc5-ad7c-3efeef4e8ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:24.844193 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bd0d4-2f0d-4bc5-ad7c-3efeef4e8ceb admin admin] PATH_INFO: `/v3/projects/18d95fd38a664ce1b69afbb11189020a/users/f2c14f2350dd450b846db0c22c43688b/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:24.860950 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691bd0d4-2f0d-4bc5-ad7c-3efeef4e8ceb admin admin] RBAC: Authorizing `identity:create_grant(project_id=18d95fd38a664ce1b69afbb11189020a, user_id=f2c14f2350dd450b846db0c22c43688b, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:42:24.863735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691bd0d4-2f0d-4bc5-ad7c-3efeef4e8ceb admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:42:24.881204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1218/2415] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:42:24 2022] PUT /identity/v3/projects/18d95fd38a664ce1b69afbb11189020a/users/f2c14f2350dd450b846db0c22c43688b/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:42:24.886898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d45474-2161-4224-adf8-29afe59c6a76 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:24.887185 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d45474-2161-4224-adf8-29afe59c6a76 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:24.887473 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d45474-2161-4224-adf8-29afe59c6a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:24.910044 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-05d45474-2161-4224-adf8-29afe59c6a76 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:42:24.911340 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-05d45474-2161-4224-adf8-29afe59c6a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:42:24.977992 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1198/2416] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:42:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:42:24.988928 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bd0d4-2f0d-4bc5-ad7c-3efeef4e8ceb admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:42:24.990962 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-691bd0d4-2f0d-4bc5-ad7c-3efeef4e8ceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:42:25.054608 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bd0d4-2f0d-4bc5-ad7c-3efeef4e8ceb admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:42:25.058540 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d4c84-51ad-4bbd-b063-5eef125b9616 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:42:25.059194 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d4c84-51ad-4bbd-b063-5eef125b9616 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:25.059479 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d4c84-51ad-4bbd-b063-5eef125b9616 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:25.059757 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d4c84-51ad-4bbd-b063-5eef125b9616 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:25.070589 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7d4c84-51ad-4bbd-b063-5eef125b9616 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:42:25.072272 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7d4c84-51ad-4bbd-b063-5eef125b9616 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:42:25.083143 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1219/2417] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:42:24 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:42:29.529453 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d8a53-3477-4c4c-889c-c6b73bbe6b9f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:29.530418 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d8a53-3477-4c4c-889c-c6b73bbe6b9f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:29.530418 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d8a53-3477-4c4c-889c-c6b73bbe6b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:29.571713 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-860d8a53-3477-4c4c-889c-c6b73bbe6b9f None None] MFA Rules not processed for user `f2c14f2350dd450b846db0c22c43688b`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:42:29.573667 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-860d8a53-3477-4c4c-889c-c6b73bbe6b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:42:29.591518 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:42:29.663633 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1199/2418] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:42:29 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:42:29.680776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d4c84-51ad-4bbd-b063-5eef125b9616 service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:42:29.683210 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-2a7d4c84-51ad-4bbd-b063-5eef125b9616 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:42:29.777893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d4c84-51ad-4bbd-b063-5eef125b9616 service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:42:29.787523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d631b0-a562-4621-83bd-bc7aa670aee0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:42:29.788202 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d631b0-a562-4621-83bd-bc7aa670aee0 service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:29.788596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d631b0-a562-4621-83bd-bc7aa670aee0 service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:29.789165 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d631b0-a562-4621-83bd-bc7aa670aee0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:29.798697 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d631b0-a562-4621-83bd-bc7aa670aee0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:42:29.800420 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d631b0-a562-4621-83bd-bc7aa670aee0 service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:42:29.810967 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1220/2419] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:42:29 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:42:31.781259 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe8403e-eda0-4868-8818-39c9d2be98c7 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:31.781920 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe8403e-eda0-4868-8818-39c9d2be98c7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:31.781920 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe8403e-eda0-4868-8818-39c9d2be98c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:31.803707 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ffe8403e-eda0-4868-8818-39c9d2be98c7 None None] MFA Rules not processed for user `f2c14f2350dd450b846db0c22c43688b`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:42:31.805114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ffe8403e-eda0-4868-8818-39c9d2be98c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:42:31.812643 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1200/2420] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:42:31 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:42:39.805854 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=75,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:42:41.797609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=25 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:42:43.467829 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d631b0-a562-4621-83bd-bc7aa670aee0 service nova] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:42:43.469462 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-79d631b0-a562-4621-83bd-bc7aa670aee0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:42:43.557098 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d631b0-a562-4621-83bd-bc7aa670aee0 service nova] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:42:43.561098 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf1c9a2-fd3f-414d-b62f-720b9638e89e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:42:43.561753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1c9a2-fd3f-414d-b62f-720b9638e89e service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:43.562037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1c9a2-fd3f-414d-b62f-720b9638e89e service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:43.562349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1c9a2-fd3f-414d-b62f-720b9638e89e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:43.565002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-3cf1c9a2-fd3f-414d-b62f-720b9638e89e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:42:43.579533 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf1c9a2-fd3f-414d-b62f-720b9638e89e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:42:43.581057 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf1c9a2-fd3f-414d-b62f-720b9638e89e service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:42:43.593289 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1221/2421] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:42:43 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:42:47.296962 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe8403e-eda0-4868-8818-39c9d2be98c7 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:42:47.309063 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe8403e-eda0-4868-8818-39c9d2be98c7 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:42:47.313171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d56de-0ba8-4419-b263-33e84f339b5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:42:47.314004 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d56de-0ba8-4419-b263-33e84f339b5f service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:47.314340 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d56de-0ba8-4419-b263-33e84f339b5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:47.314687 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d56de-0ba8-4419-b263-33e84f339b5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:47.316869 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-377d56de-0ba8-4419-b263-33e84f339b5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:42:47.335326 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d56de-0ba8-4419-b263-33e84f339b5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:42:47.337232 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d56de-0ba8-4419-b263-33e84f339b5f service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:42:47.355999 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1201/2422] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:42:47 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:42:53.478014 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5012d-f11f-4450-b696-727cbf62b832 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:53.478605 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5012d-f11f-4450-b696-727cbf62b832 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:53.478605 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5012d-f11f-4450-b696-727cbf62b832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:53.523647 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-29a5012d-f11f-4450-b696-727cbf62b832 None None] MFA Rules not processed for user `41a38b5af1654f44bc6cb5832c2e8e87`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:42:53.524972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-29a5012d-f11f-4450-b696-727cbf62b832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:42:53.586096 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1222/2423] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:42:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:42:57.349592 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=6 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:42:58.059222 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d56de-0ba8-4419-b263-33e84f339b5f service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:42:58.075410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d56de-0ba8-4419-b263-33e84f339b5f service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:42:58.081608 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f7678-b4fc-4457-bd01-c9c4971c4830 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:42:58.082269 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f7678-b4fc-4457-bd01-c9c4971c4830 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:42:58.082559 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f7678-b4fc-4457-bd01-c9c4971c4830 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:42:58.082838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f7678-b4fc-4457-bd01-c9c4971c4830 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:42:58.084731 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-513f7678-b4fc-4457-bd01-c9c4971c4830 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:42:58.101664 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f7678-b4fc-4457-bd01-c9c4971c4830 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:42:58.103286 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f7678-b4fc-4457-bd01-c9c4971c4830 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:42:58.126381 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1202/2424] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:42:58 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:43:03.577869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=69 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:43:08.122657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=10 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:43:52.760986 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5012d-f11f-4450-b696-727cbf62b832 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:43:52.776305 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5012d-f11f-4450-b696-727cbf62b832 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:43:52.782587 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae640f-284f-4954-9e9b-f133e94d41c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:43:52.783204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae640f-284f-4954-9e9b-f133e94d41c5 service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:43:52.783486 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae640f-284f-4954-9e9b-f133e94d41c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:43:52.783761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae640f-284f-4954-9e9b-f133e94d41c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:43:52.785705 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d6ae640f-284f-4954-9e9b-f133e94d41c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:43:52.875213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae640f-284f-4954-9e9b-f133e94d41c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:43:52.876789 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae640f-284f-4954-9e9b-f133e94d41c5 service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:43:52.895531 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1223/2425] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:43:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:43:54.381182 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f7678-b4fc-4457-bd01-c9c4971c4830 service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:43:54.391986 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f7678-b4fc-4457-bd01-c9c4971c4830 service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:43:54.395876 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff8ca6-deea-4c6c-bad4-3036a9396c34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:43:54.396667 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff8ca6-deea-4c6c-bad4-3036a9396c34 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:43:54.396946 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff8ca6-deea-4c6c-bad4-3036a9396c34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:43:54.397286 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff8ca6-deea-4c6c-bad4-3036a9396c34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:43:54.399249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-33ff8ca6-deea-4c6c-bad4-3036a9396c34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:43:54.415254 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff8ca6-deea-4c6c-bad4-3036a9396c34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:43:54.417667 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff8ca6-deea-4c6c-bad4-3036a9396c34 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:43:54.431561 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1203/2426] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:43:54 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:44:02.635800 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae640f-284f-4954-9e9b-f133e94d41c5 service nova] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:02.638376 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d6ae640f-284f-4954-9e9b-f133e94d41c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:02.658925 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae640f-284f-4954-9e9b-f133e94d41c5 service nova] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:02.665392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3fadd1-3ca3-4655-a45f-72693010a4cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:02.666047 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3fadd1-3ca3-4655-a45f-72693010a4cd service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:02.666329 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3fadd1-3ca3-4655-a45f-72693010a4cd service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:02.666600 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3fadd1-3ca3-4655-a45f-72693010a4cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:02.668511 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-db3fadd1-3ca3-4655-a45f-72693010a4cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:02.684992 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3fadd1-3ca3-4655-a45f-72693010a4cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:02.688437 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3fadd1-3ca3-4655-a45f-72693010a4cd service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:02.701460 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1224/2427] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:44:02 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:44:04.425347 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=6 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:44:05.412892 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff8ca6-deea-4c6c-bad4-3036a9396c34 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:05.428984 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff8ca6-deea-4c6c-bad4-3036a9396c34 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:05.435509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c1a573-1a70-46a3-9c44-2fc212ec6f75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:05.436374 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1a573-1a70-46a3-9c44-2fc212ec6f75 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:05.436812 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1a573-1a70-46a3-9c44-2fc212ec6f75 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:05.437092 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1a573-1a70-46a3-9c44-2fc212ec6f75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:05.438962 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d4c1a573-1a70-46a3-9c44-2fc212ec6f75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:05.454255 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c1a573-1a70-46a3-9c44-2fc212ec6f75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:05.456362 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c1a573-1a70-46a3-9c44-2fc212ec6f75 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:05.468606 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1204/2428] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:44:05 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:44:06.863209 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3fadd1-3ca3-4655-a45f-72693010a4cd service nova] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:06.864859 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-db3fadd1-3ca3-4655-a45f-72693010a4cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:06.876915 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3fadd1-3ca3-4655-a45f-72693010a4cd service nova] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:06.880765 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d1e2f-96bc-4be7-bcd4-213e9f7450e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:06.881418 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d1e2f-96bc-4be7-bcd4-213e9f7450e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:06.881693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d1e2f-96bc-4be7-bcd4-213e9f7450e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:06.882332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d1e2f-96bc-4be7-bcd4-213e9f7450e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:06.884040 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-8a9d1e2f-96bc-4be7-bcd4-213e9f7450e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:06.900117 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d1e2f-96bc-4be7-bcd4-213e9f7450e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:06.900117 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d1e2f-96bc-4be7-bcd4-213e9f7450e2 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:06.914767 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1225/2429] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:44:06 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:44:13.860880 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c1a573-1a70-46a3-9c44-2fc212ec6f75 service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:13.863084 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d4c1a573-1a70-46a3-9c44-2fc212ec6f75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:13.882978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c1a573-1a70-46a3-9c44-2fc212ec6f75 service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:13.888558 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02892731-9d6f-43b1-92d0-387f9e845d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:13.889262 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02892731-9d6f-43b1-92d0-387f9e845d91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:13.889611 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02892731-9d6f-43b1-92d0-387f9e845d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:13.889898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02892731-9d6f-43b1-92d0-387f9e845d91 admin admin] PATH_INFO: `/v3/users/6921033062424e14bc30f9b94bf911db` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:13.891269 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02892731-9d6f-43b1-92d0-387f9e845d91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6921033062424e14bc30f9b94bf911db)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:13.893298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02892731-9d6f-43b1-92d0-387f9e845d91 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:13.962094 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-02892731-9d6f-43b1-92d0-387f9e845d91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6921033062424e14bc30f9b94bf911db'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:13.975214 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-02892731-9d6f-43b1-92d0-387f9e845d91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6921033062424e14bc30f9b94bf911db'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:13.976135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-02892731-9d6f-43b1-92d0-387f9e845d91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6921033062424e14bc30f9b94bf911db'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:13.990401 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-02892731-9d6f-43b1-92d0-387f9e845d91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6921033062424e14bc30f9b94bf911db'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:13.991209 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-02892731-9d6f-43b1-92d0-387f9e845d91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6921033062424e14bc30f9b94bf911db'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:14.006962 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1205/2430] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:44:13 2022] DELETE /identity/v3/users/6921033062424e14bc30f9b94bf911db => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:44:14.209877 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d1e2f-96bc-4be7-bcd4-213e9f7450e2 service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:14.220230 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=54 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:44:14.229830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d1e2f-96bc-4be7-bcd4-213e9f7450e2 service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:14.299606 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7c4de-d886-43fb-bc75-f6258c4f1caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:14.301291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7c4de-d886-43fb-bc75-f6258c4f1caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:14.301686 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7c4de-d886-43fb-bc75-f6258c4f1caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:14.301977 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7c4de-d886-43fb-bc75-f6258c4f1caf admin admin] PATH_INFO: `/v3/projects/bee0d982e3ba4f7dba65f51636856b5a` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:14.303726 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7c4de-d886-43fb-bc75-f6258c4f1caf admin admin] RBAC: Authorizing `identity:delete_project(project_id=bee0d982e3ba4f7dba65f51636856b5a)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:14.306197 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7c4de-d886-43fb-bc75-f6258c4f1caf admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:14.376306 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-bca7c4de-d886-43fb-bc75-f6258c4f1caf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bee0d982e3ba4f7dba65f51636856b5a', 'request_id': 'req-bca7c4de-d886-43fb-bc75-f6258c4f1caf'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:14.386578 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-bca7c4de-d886-43fb-bc75-f6258c4f1caf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bee0d982e3ba4f7dba65f51636856b5a', 'request_id': 'req-bca7c4de-d886-43fb-bc75-f6258c4f1caf'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:14.394679 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-bca7c4de-d886-43fb-bc75-f6258c4f1caf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bee0d982e3ba4f7dba65f51636856b5a', 'request_id': 'req-bca7c4de-d886-43fb-bc75-f6258c4f1caf'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:14.410150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-bca7c4de-d886-43fb-bc75-f6258c4f1caf admin admin] The token cache is being invalidate because project bee0d982e3ba4f7dba65f51636856b5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:44:14.410455 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-bca7c4de-d886-43fb-bc75-f6258c4f1caf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:14.411196 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-bca7c4de-d886-43fb-bc75-f6258c4f1caf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:14.412898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1226/2431] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:44:14 2022] DELETE /identity/v3/projects/bee0d982e3ba4f7dba65f51636856b5a => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:44:14.419087 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02892731-9d6f-43b1-92d0-387f9e845d91 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:14.420879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-02892731-9d6f-43b1-92d0-387f9e845d91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:14.436912 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02892731-9d6f-43b1-92d0-387f9e845d91 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:14.506290 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a859e0d-cb9e-4e3e-8fe0-19b8dd1d62f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:14.506829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a859e0d-cb9e-4e3e-8fe0-19b8dd1d62f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:14.507121 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a859e0d-cb9e-4e3e-8fe0-19b8dd1d62f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:14.507409 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a859e0d-cb9e-4e3e-8fe0-19b8dd1d62f6 admin admin] PATH_INFO: `/v3/users/3fe3e0b894ae428d9a9a52377902e062` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:14.508696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a859e0d-cb9e-4e3e-8fe0-19b8dd1d62f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fe3e0b894ae428d9a9a52377902e062)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:14.510660 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a859e0d-cb9e-4e3e-8fe0-19b8dd1d62f6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:14.569661 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6a859e0d-cb9e-4e3e-8fe0-19b8dd1d62f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fe3e0b894ae428d9a9a52377902e062'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:14.579877 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6a859e0d-cb9e-4e3e-8fe0-19b8dd1d62f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fe3e0b894ae428d9a9a52377902e062'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:14.581021 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6a859e0d-cb9e-4e3e-8fe0-19b8dd1d62f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fe3e0b894ae428d9a9a52377902e062'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:14.588721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:44:14.603201 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6a859e0d-cb9e-4e3e-8fe0-19b8dd1d62f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fe3e0b894ae428d9a9a52377902e062'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:14.604069 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6a859e0d-cb9e-4e3e-8fe0-19b8dd1d62f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fe3e0b894ae428d9a9a52377902e062'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:14.620364 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1206/2432] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:44:14 2022] DELETE /identity/v3/users/3fe3e0b894ae428d9a9a52377902e062 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:44:14.853770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7c4de-d886-43fb-bc75-f6258c4f1caf admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:14.855665 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-bca7c4de-d886-43fb-bc75-f6258c4f1caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:14.875282 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7c4de-d886-43fb-bc75-f6258c4f1caf admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:14.941299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea93331-bcfc-4b1b-930e-b4823ca4b61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:14.941948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea93331-bcfc-4b1b-930e-b4823ca4b61f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:14.942429 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea93331-bcfc-4b1b-930e-b4823ca4b61f admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:14.942540 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea93331-bcfc-4b1b-930e-b4823ca4b61f admin admin] PATH_INFO: `/v3/projects/fc5257a153a444dfbf0c14f378bd88c5` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:14.943733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea93331-bcfc-4b1b-930e-b4823ca4b61f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc5257a153a444dfbf0c14f378bd88c5)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:14.945729 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea93331-bcfc-4b1b-930e-b4823ca4b61f admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:14.985481 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:44:14.996300 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-bea93331-bcfc-4b1b-930e-b4823ca4b61f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc5257a153a444dfbf0c14f378bd88c5', 'request_id': 'req-bea93331-bcfc-4b1b-930e-b4823ca4b61f'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:15.005877 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-bea93331-bcfc-4b1b-930e-b4823ca4b61f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc5257a153a444dfbf0c14f378bd88c5', 'request_id': 'req-bea93331-bcfc-4b1b-930e-b4823ca4b61f'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:15.012139 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-bea93331-bcfc-4b1b-930e-b4823ca4b61f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc5257a153a444dfbf0c14f378bd88c5', 'request_id': 'req-bea93331-bcfc-4b1b-930e-b4823ca4b61f'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:15.024811 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-bea93331-bcfc-4b1b-930e-b4823ca4b61f admin admin] The token cache is being invalidate because project fc5257a153a444dfbf0c14f378bd88c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:44:15.025135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-bea93331-bcfc-4b1b-930e-b4823ca4b61f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:15.025826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-bea93331-bcfc-4b1b-930e-b4823ca4b61f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:15.027579 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1227/2433] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:44:14 2022] DELETE /identity/v3/projects/fc5257a153a444dfbf0c14f378bd88c5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:44:24.616684 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=3,keystone:SELECT=6,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:44:25.021105 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:DELETE=5,keystone:SELECT=11,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:44:26.569037 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a859e0d-cb9e-4e3e-8fe0-19b8dd1d62f6 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:26.570583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6a859e0d-cb9e-4e3e-8fe0-19b8dd1d62f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:26.635249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a859e0d-cb9e-4e3e-8fe0-19b8dd1d62f6 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:26.688962 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547598cb-0bec-4971-af87-8e7bd0444ed2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:26.689716 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547598cb-0bec-4971-af87-8e7bd0444ed2 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:26.690065 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547598cb-0bec-4971-af87-8e7bd0444ed2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:26.690418 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547598cb-0bec-4971-af87-8e7bd0444ed2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:26.693134 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-547598cb-0bec-4971-af87-8e7bd0444ed2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:26.768949 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547598cb-0bec-4971-af87-8e7bd0444ed2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:26.770335 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547598cb-0bec-4971-af87-8e7bd0444ed2 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:26.782724 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1207/2434] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:44:26 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:44:30.088868 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea93331-bcfc-4b1b-930e-b4823ca4b61f admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:30.090567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-bea93331-bcfc-4b1b-930e-b4823ca4b61f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:30.151767 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547598cb-0bec-4971-af87-8e7bd0444ed2 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:30.153176 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-547598cb-0bec-4971-af87-8e7bd0444ed2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:30.170351 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547598cb-0bec-4971-af87-8e7bd0444ed2 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:30.176561 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ee5b7-ada3-4fad-9fe7-4064bd120d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:30.177372 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ee5b7-ada3-4fad-9fe7-4064bd120d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:30.177657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ee5b7-ada3-4fad-9fe7-4064bd120d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:30.177977 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ee5b7-ada3-4fad-9fe7-4064bd120d67 admin admin] PATH_INFO: `/v3/users/3aa2e2a5a6ed4c5c9db91269266dd30b` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:30.179304 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ee5b7-ada3-4fad-9fe7-4064bd120d67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aa2e2a5a6ed4c5c9db91269266dd30b)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:30.181507 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ee5b7-ada3-4fad-9fe7-4064bd120d67 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:30.186935 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea93331-bcfc-4b1b-930e-b4823ca4b61f admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:30.192877 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a61b2b-8fda-4f02-a0e4-a068f86a4c38 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c7cfbe911c4378857848966f4f8d98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:30.193512 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a61b2b-8fda-4f02-a0e4-a068f86a4c38 service placement] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:30.193981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a61b2b-8fda-4f02-a0e4-a068f86a4c38 service placement] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:30.194261 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a61b2b-8fda-4f02-a0e4-a068f86a4c38 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:30.196454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-b7a61b2b-8fda-4f02-a0e4-a068f86a4c38 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:30.221212 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a61b2b-8fda-4f02-a0e4-a068f86a4c38 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:30.226420 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a61b2b-8fda-4f02-a0e4-a068f86a4c38 service placement] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:30.248153 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1228/2435] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:44:30 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:44:30.260256 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bc1ee5b7-ada3-4fad-9fe7-4064bd120d67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aa2e2a5a6ed4c5c9db91269266dd30b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:30.272525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bc1ee5b7-ada3-4fad-9fe7-4064bd120d67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aa2e2a5a6ed4c5c9db91269266dd30b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:30.273396 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bc1ee5b7-ada3-4fad-9fe7-4064bd120d67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aa2e2a5a6ed4c5c9db91269266dd30b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:30.280428 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:44:30.290894 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bc1ee5b7-ada3-4fad-9fe7-4064bd120d67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aa2e2a5a6ed4c5c9db91269266dd30b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:30.291799 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bc1ee5b7-ada3-4fad-9fe7-4064bd120d67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aa2e2a5a6ed4c5c9db91269266dd30b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:30.306903 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1208/2436] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:44:30 2022] DELETE /identity/v3/users/3aa2e2a5a6ed4c5c9db91269266dd30b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:44:30.502415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a61b2b-8fda-4f02-a0e4-a068f86a4c38 service placement] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:30.519466 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a61b2b-8fda-4f02-a0e4-a068f86a4c38 service placement] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:30.592633 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916da4b-9bff-483d-9bad-1800c2805551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:30.593358 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916da4b-9bff-483d-9bad-1800c2805551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:30.593646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916da4b-9bff-483d-9bad-1800c2805551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:30.593924 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916da4b-9bff-483d-9bad-1800c2805551 admin admin] PATH_INFO: `/v3/projects/6f46ab11dd0b428486c673afc14b5b63` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:30.595188 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916da4b-9bff-483d-9bad-1800c2805551 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f46ab11dd0b428486c673afc14b5b63)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:30.597538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916da4b-9bff-483d-9bad-1800c2805551 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:30.653774 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7916da4b-9bff-483d-9bad-1800c2805551 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f46ab11dd0b428486c673afc14b5b63', 'request_id': 'req-7916da4b-9bff-483d-9bad-1800c2805551'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:30.663515 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7916da4b-9bff-483d-9bad-1800c2805551 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f46ab11dd0b428486c673afc14b5b63', 'request_id': 'req-7916da4b-9bff-483d-9bad-1800c2805551'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:30.669607 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7916da4b-9bff-483d-9bad-1800c2805551 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f46ab11dd0b428486c673afc14b5b63', 'request_id': 'req-7916da4b-9bff-483d-9bad-1800c2805551'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:30.681597 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7916da4b-9bff-483d-9bad-1800c2805551 admin admin] The token cache is being invalidate because project 6f46ab11dd0b428486c673afc14b5b63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:44:30.681885 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7916da4b-9bff-483d-9bad-1800c2805551 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:30.682779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-7916da4b-9bff-483d-9bad-1800c2805551 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:30.684471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1229/2437] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:44:30 2022] DELETE /identity/v3/projects/6f46ab11dd0b428486c673afc14b5b63 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:44:30.689954 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc125f-4c24-47a9-891f-2edce6397dc7 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:30.690276 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc125f-4c24-47a9-891f-2edce6397dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:30.690556 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc125f-4c24-47a9-891f-2edce6397dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:30.713124 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b8fc125f-4c24-47a9-891f-2edce6397dc7 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:44:30.714441 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b8fc125f-4c24-47a9-891f-2edce6397dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:30.784051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1209/2438] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:44:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:44:30.793940 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a7ee4-020a-466e-803d-d048c2cc7366 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:30.794237 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a7ee4-020a-466e-803d-d048c2cc7366 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:30.794521 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a7ee4-020a-466e-803d-d048c2cc7366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:30.821462 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-d52a7ee4-020a-466e-803d-d048c2cc7366 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:44:30.822666 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d52a7ee4-020a-466e-803d-d048c2cc7366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:30.832598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1230/2439] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:44:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:44:30.837278 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc125f-4c24-47a9-891f-2edce6397dc7 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:30.848293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc125f-4c24-47a9-891f-2edce6397dc7 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:30.853518 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561400c2-320a-4363-b0df-db888a2087c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:30.855297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561400c2-320a-4363-b0df-db888a2087c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:30.855297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561400c2-320a-4363-b0df-db888a2087c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:30.855297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561400c2-320a-4363-b0df-db888a2087c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:30.855606 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561400c2-320a-4363-b0df-db888a2087c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:44:30.856140 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561400c2-320a-4363-b0df-db888a2087c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:30.859414 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561400c2-320a-4363-b0df-db888a2087c9 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:30.871911 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1210/2440] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:44:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:44:30.876656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a7ee4-020a-466e-803d-d048c2cc7366 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:30.889133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a7ee4-020a-466e-803d-d048c2cc7366 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:30.894575 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d1590-a71a-492a-b088-f7932f3ab184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:30.895329 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d1590-a71a-492a-b088-f7932f3ab184 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:30.895691 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d1590-a71a-492a-b088-f7932f3ab184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:30.896170 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d1590-a71a-492a-b088-f7932f3ab184 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:30.896873 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d1590-a71a-492a-b088-f7932f3ab184 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:30.899166 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d1590-a71a-492a-b088-f7932f3ab184 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:30.914180 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:44:30.959124 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1231/2441] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:44:30 2022] POST /identity/v3/projects => generated 368 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:44:30.965077 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561400c2-320a-4363-b0df-db888a2087c9 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:30.988533 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561400c2-320a-4363-b0df-db888a2087c9 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:31.053580 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfef23-c94b-4729-98ef-a28796e5b9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:31.054466 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfef23-c94b-4729-98ef-a28796e5b9de admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:31.054823 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfef23-c94b-4729-98ef-a28796e5b9de admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:31.055135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfef23-c94b-4729-98ef-a28796e5b9de admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:31.055821 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcfef23-c94b-4729-98ef-a28796e5b9de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:31.057890 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcfef23-c94b-4729-98ef-a28796e5b9de admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:31.086827 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1211/2442] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:44:30 2022] POST /identity/v3/users => generated 285 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:44:31.091434 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d1590-a71a-492a-b088-f7932f3ab184 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:31.103447 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d1590-a71a-492a-b088-f7932f3ab184 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:31.108716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3708852-bdb9-491c-8c34-958c357b1cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:31.109365 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3708852-bdb9-491c-8c34-958c357b1cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:31.109687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3708852-bdb9-491c-8c34-958c357b1cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:31.109952 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3708852-bdb9-491c-8c34-958c357b1cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:31.110669 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3708852-bdb9-491c-8c34-958c357b1cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:31.112381 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3708852-bdb9-491c-8c34-958c357b1cd0 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:31.123568 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1232/2443] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:44:31 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:44:31.128319 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfef23-c94b-4729-98ef-a28796e5b9de admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:31.139164 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfef23-c94b-4729-98ef-a28796e5b9de admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:31.144213 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3f025-f3cd-49c7-a4a1-d1305d7602ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:31.144845 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3f025-f3cd-49c7-a4a1-d1305d7602ba admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:31.145173 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3f025-f3cd-49c7-a4a1-d1305d7602ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:31.145412 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3f025-f3cd-49c7-a4a1-d1305d7602ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:31.146133 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3f025-f3cd-49c7-a4a1-d1305d7602ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:31.147825 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3f025-f3cd-49c7-a4a1-d1305d7602ba admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:31.159772 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1212/2444] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:44:31 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:44:31.166227 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3708852-bdb9-491c-8c34-958c357b1cd0 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:31.180751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3708852-bdb9-491c-8c34-958c357b1cd0 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:31.186014 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a0341-aef2-48b4-aa46-213e07a5d6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:31.186635 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a0341-aef2-48b4-aa46-213e07a5d6e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:31.186926 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a0341-aef2-48b4-aa46-213e07a5d6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:31.187201 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a0341-aef2-48b4-aa46-213e07a5d6e7 admin admin] PATH_INFO: `/v3/projects/48266cb993ca43fd90a7c8ab759d57d9/users/80d65ad8a90f477596fa31549de108c4/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:31.202756 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a0341-aef2-48b4-aa46-213e07a5d6e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48266cb993ca43fd90a7c8ab759d57d9, user_id=80d65ad8a90f477596fa31549de108c4, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:31.205507 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a0341-aef2-48b4-aa46-213e07a5d6e7 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:31.222175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1233/2445] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:44:31 2022] PUT /identity/v3/projects/48266cb993ca43fd90a7c8ab759d57d9/users/80d65ad8a90f477596fa31549de108c4/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:44:31.233603 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48714e00-4028-41f9-93e6-cfd888a7e7bb None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:31.233916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48714e00-4028-41f9-93e6-cfd888a7e7bb None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:31.234195 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48714e00-4028-41f9-93e6-cfd888a7e7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:31.263600 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-48714e00-4028-41f9-93e6-cfd888a7e7bb None None] MFA Rules not processed for user `80d65ad8a90f477596fa31549de108c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:44:31.264897 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-48714e00-4028-41f9-93e6-cfd888a7e7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:31.308790 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:44:31.355425 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1213/2446] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:44:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:44:31.369631 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a0341-aef2-48b4-aa46-213e07a5d6e7 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:31.372249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-b42a0341-aef2-48b4-aa46-213e07a5d6e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:31.445708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a0341-aef2-48b4-aa46-213e07a5d6e7 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:31.450317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b7920-ba38-4bd8-9293-12ce48b26bb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:31.450955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b7920-ba38-4bd8-9293-12ce48b26bb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:31.451234 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b7920-ba38-4bd8-9293-12ce48b26bb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:31.451534 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b7920-ba38-4bd8-9293-12ce48b26bb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:31.461111 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634b7920-ba38-4bd8-9293-12ce48b26bb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:31.462582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634b7920-ba38-4bd8-9293-12ce48b26bb5 service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:31.472462 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1234/2447] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:44:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:44:39.161065 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48714e00-4028-41f9-93e6-cfd888a7e7bb None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:39.163606 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-48714e00-4028-41f9-93e6-cfd888a7e7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:39.258422 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48714e00-4028-41f9-93e6-cfd888a7e7bb None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:39.266347 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff80f89-2cb5-4960-b8ff-2a07d19748a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:39.267597 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff80f89-2cb5-4960-b8ff-2a07d19748a8 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:39.267597 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff80f89-2cb5-4960-b8ff-2a07d19748a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:39.267597 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff80f89-2cb5-4960-b8ff-2a07d19748a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:39.270783 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4ff80f89-2cb5-4960-b8ff-2a07d19748a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:39.286194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff80f89-2cb5-4960-b8ff-2a07d19748a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:39.287925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff80f89-2cb5-4960-b8ff-2a07d19748a8 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:39.300610 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1214/2448] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:44:39 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:44:40.123044 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b7920-ba38-4bd8-9293-12ce48b26bb5 service cinder] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:40.193465 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b7920-ba38-4bd8-9293-12ce48b26bb5 service cinder] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:40.199064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a22e4-068d-454d-8684-4b3e98ccc6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:40.199749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a22e4-068d-454d-8684-4b3e98ccc6e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:40.200053 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a22e4-068d-454d-8684-4b3e98ccc6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:40.200336 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a22e4-068d-454d-8684-4b3e98ccc6e8 admin admin] PATH_INFO: `/v3/users/80d65ad8a90f477596fa31549de108c4` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:40.201579 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a22e4-068d-454d-8684-4b3e98ccc6e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80d65ad8a90f477596fa31549de108c4)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:40.203580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a22e4-068d-454d-8684-4b3e98ccc6e8 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:40.261334 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e68a22e4-068d-454d-8684-4b3e98ccc6e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80d65ad8a90f477596fa31549de108c4'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:40.262632 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e68a22e4-068d-454d-8684-4b3e98ccc6e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80d65ad8a90f477596fa31549de108c4'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:40.274146 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:44:40.287447 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e68a22e4-068d-454d-8684-4b3e98ccc6e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80d65ad8a90f477596fa31549de108c4'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:40.296929 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e68a22e4-068d-454d-8684-4b3e98ccc6e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80d65ad8a90f477596fa31549de108c4'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:40.307852 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-e68a22e4-068d-454d-8684-4b3e98ccc6e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80d65ad8a90f477596fa31549de108c4'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:40.314911 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1235/2449] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:44:40 2022] DELETE /identity/v3/users/80d65ad8a90f477596fa31549de108c4 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:44:40.319837 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5476f-ed5b-4eba-b575-934bb038dbb8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:40.320554 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5476f-ed5b-4eba-b575-934bb038dbb8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:40.320554 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5476f-ed5b-4eba-b575-934bb038dbb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:40.342091 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-09b5476f-ed5b-4eba-b575-934bb038dbb8 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:44:40.343383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-09b5476f-ed5b-4eba-b575-934bb038dbb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:40.410813 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1215/2450] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:44:40 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:44:40.421522 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a22e4-068d-454d-8684-4b3e98ccc6e8 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:40.423463 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-e68a22e4-068d-454d-8684-4b3e98ccc6e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:40.515695 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a22e4-068d-454d-8684-4b3e98ccc6e8 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:40.520762 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e85f2-a476-48e6-b819-01449a43f80d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:40.521425 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e85f2-a476-48e6-b819-01449a43f80d service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:40.521712 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e85f2-a476-48e6-b819-01449a43f80d service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:40.521995 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e85f2-a476-48e6-b819-01449a43f80d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:40.533365 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e85f2-a476-48e6-b819-01449a43f80d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:40.534937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e85f2-a476-48e6-b819-01449a43f80d service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:40.551857 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1236/2451] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:44:40 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:44:40.928768 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5476f-ed5b-4eba-b575-934bb038dbb8 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:40.930543 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-09b5476f-ed5b-4eba-b575-934bb038dbb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:40.946708 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5476f-ed5b-4eba-b575-934bb038dbb8 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:40.953882 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f619a2-8689-4841-bf98-7885c6c2b7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:40.954625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f619a2-8689-4841-bf98-7885c6c2b7f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:40.954915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f619a2-8689-4841-bf98-7885c6c2b7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:40.955202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f619a2-8689-4841-bf98-7885c6c2b7f6 admin admin] PATH_INFO: `/v3/projects/48266cb993ca43fd90a7c8ab759d57d9` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:40.956699 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f619a2-8689-4841-bf98-7885c6c2b7f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48266cb993ca43fd90a7c8ab759d57d9)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:40.959270 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f619a2-8689-4841-bf98-7885c6c2b7f6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:41.035047 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d8f619a2-8689-4841-bf98-7885c6c2b7f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48266cb993ca43fd90a7c8ab759d57d9', 'request_id': 'req-d8f619a2-8689-4841-bf98-7885c6c2b7f6'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:41.042776 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d8f619a2-8689-4841-bf98-7885c6c2b7f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48266cb993ca43fd90a7c8ab759d57d9', 'request_id': 'req-d8f619a2-8689-4841-bf98-7885c6c2b7f6'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:41.044258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:44:41.051655 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d8f619a2-8689-4841-bf98-7885c6c2b7f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48266cb993ca43fd90a7c8ab759d57d9', 'request_id': 'req-d8f619a2-8689-4841-bf98-7885c6c2b7f6'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:41.071748 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d8f619a2-8689-4841-bf98-7885c6c2b7f6 admin admin] The token cache is being invalidate because project 48266cb993ca43fd90a7c8ab759d57d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:44:41.072238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d8f619a2-8689-4841-bf98-7885c6c2b7f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:41.073491 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d8f619a2-8689-4841-bf98-7885c6c2b7f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:44:41.076120 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1216/2452] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:44:40 2022] DELETE /identity/v3/projects/48266cb993ca43fd90a7c8ab759d57d9 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:44:41.083479 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504248f2-c90f-48ae-ac8c-a9a48df03558 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:41.083751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504248f2-c90f-48ae-ac8c-a9a48df03558 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:41.084392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504248f2-c90f-48ae-ac8c-a9a48df03558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:41.110229 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-504248f2-c90f-48ae-ac8c-a9a48df03558 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:44:41.111566 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-504248f2-c90f-48ae-ac8c-a9a48df03558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:41.183082 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1237/2453] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:44:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:44:41.192503 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e28a1e-eb76-431a-acf8-d212590ed371 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:41.192791 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e28a1e-eb76-431a-acf8-d212590ed371 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:41.193081 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e28a1e-eb76-431a-acf8-d212590ed371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:41.213867 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f0e28a1e-eb76-431a-acf8-d212590ed371 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:44:41.215126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f0e28a1e-eb76-431a-acf8-d212590ed371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:41.224110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1217/2454] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:44:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:44:41.236190 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504248f2-c90f-48ae-ac8c-a9a48df03558 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:41.248378 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504248f2-c90f-48ae-ac8c-a9a48df03558 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:41.254304 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71fed0a-df64-4b0c-9605-920758e76302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:41.254953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71fed0a-df64-4b0c-9605-920758e76302 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:41.255225 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71fed0a-df64-4b0c-9605-920758e76302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:41.255519 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71fed0a-df64-4b0c-9605-920758e76302 admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:41.256285 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71fed0a-df64-4b0c-9605-920758e76302 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:44:41.256596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71fed0a-df64-4b0c-9605-920758e76302 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:41.258432 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71fed0a-df64-4b0c-9605-920758e76302 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:41.276476 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1238/2455] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:44:41 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:44:41.281808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e28a1e-eb76-431a-acf8-d212590ed371 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:41.294141 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e28a1e-eb76-431a-acf8-d212590ed371 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:41.306559 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd5ed0-cfb6-4e4f-b920-4d3c07954a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:41.307351 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd5ed0-cfb6-4e4f-b920-4d3c07954a22 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:41.307713 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd5ed0-cfb6-4e4f-b920-4d3c07954a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:41.307879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd5ed0-cfb6-4e4f-b920-4d3c07954a22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:41.308624 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd5ed0-cfb6-4e4f-b920-4d3c07954a22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:41.310880 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd5ed0-cfb6-4e4f-b920-4d3c07954a22 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:41.346847 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1218/2456] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:44:41 2022] POST /identity/v3/projects => generated 374 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:44:41.351423 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71fed0a-df64-4b0c-9605-920758e76302 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:41.364043 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71fed0a-df64-4b0c-9605-920758e76302 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:41.434904 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df401d89-2718-4320-96ba-dd5789c137b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:41.435498 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df401d89-2718-4320-96ba-dd5789c137b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:41.435809 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df401d89-2718-4320-96ba-dd5789c137b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:41.436119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df401d89-2718-4320-96ba-dd5789c137b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:41.436864 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df401d89-2718-4320-96ba-dd5789c137b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:41.438961 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df401d89-2718-4320-96ba-dd5789c137b6 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:41.447382 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:44:41.464582 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1239/2457] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:44:41 2022] POST /identity/v3/users => generated 288 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:44:41.469178 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd5ed0-cfb6-4e4f-b920-4d3c07954a22 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:41.481637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd5ed0-cfb6-4e4f-b920-4d3c07954a22 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:41.487612 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e49ac-66dd-4c10-a0eb-eb42c12204a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:41.488334 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e49ac-66dd-4c10-a0eb-eb42c12204a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:41.488628 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e49ac-66dd-4c10-a0eb-eb42c12204a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:41.488910 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e49ac-66dd-4c10-a0eb-eb42c12204a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:41.489642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e49ac-66dd-4c10-a0eb-eb42c12204a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:41.491295 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e49ac-66dd-4c10-a0eb-eb42c12204a7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:41.503444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1219/2458] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:44:41 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:44:41.508883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df401d89-2718-4320-96ba-dd5789c137b6 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:41.523794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df401d89-2718-4320-96ba-dd5789c137b6 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:41.529856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eefff18-ee20-4a08-b3f6-2be76dbc5f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:41.530563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eefff18-ee20-4a08-b3f6-2be76dbc5f80 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:41.530851 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eefff18-ee20-4a08-b3f6-2be76dbc5f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:41.531130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eefff18-ee20-4a08-b3f6-2be76dbc5f80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:41.531888 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eefff18-ee20-4a08-b3f6-2be76dbc5f80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:41.533843 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eefff18-ee20-4a08-b3f6-2be76dbc5f80 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:41.546384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1240/2459] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:44:41 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:44:41.551089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e49ac-66dd-4c10-a0eb-eb42c12204a7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:41.564460 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e49ac-66dd-4c10-a0eb-eb42c12204a7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:41.573802 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef96b88-da42-482c-a9d1-5c17389057e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:41.574877 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef96b88-da42-482c-a9d1-5c17389057e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:41.575258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef96b88-da42-482c-a9d1-5c17389057e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:41.575656 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef96b88-da42-482c-a9d1-5c17389057e5 admin admin] PATH_INFO: `/v3/projects/7b99cd04920d411984cf8efcc9458967/users/94bd01a2aa764b6583fd477015f56c82/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:41.596898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef96b88-da42-482c-a9d1-5c17389057e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b99cd04920d411984cf8efcc9458967, user_id=94bd01a2aa764b6583fd477015f56c82, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:41.599641 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef96b88-da42-482c-a9d1-5c17389057e5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:41.626487 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1220/2460] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:44:41 2022] PUT /identity/v3/projects/7b99cd04920d411984cf8efcc9458967/users/94bd01a2aa764b6583fd477015f56c82/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:44:41.628921 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b14b3c-0c75-41f9-8be1-b58bc4d7c4ef None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:41.629284 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b14b3c-0c75-41f9-8be1-b58bc4d7c4ef None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:41.629636 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b14b3c-0c75-41f9-8be1-b58bc4d7c4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:41.667315 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-d4b14b3c-0c75-41f9-8be1-b58bc4d7c4ef None None] MFA Rules not processed for user `94bd01a2aa764b6583fd477015f56c82`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:44:41.668707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d4b14b3c-0c75-41f9-8be1-b58bc4d7c4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:41.742078 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1241/2461] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:44:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:44:41.747988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef96b88-da42-482c-a9d1-5c17389057e5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:41.814955 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef96b88-da42-482c-a9d1-5c17389057e5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:41.820695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936c2ff-949e-4589-8792-fa5a43fd7d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:41.821334 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936c2ff-949e-4589-8792-fa5a43fd7d8f admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:41.821650 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936c2ff-949e-4589-8792-fa5a43fd7d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:41.822045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936c2ff-949e-4589-8792-fa5a43fd7d8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:41.822737 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936c2ff-949e-4589-8792-fa5a43fd7d8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:41.824784 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936c2ff-949e-4589-8792-fa5a43fd7d8f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:41.856102 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1221/2462] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:44:41 2022] POST /identity/v3/projects => generated 374 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:44:41.863494 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b14b3c-0c75-41f9-8be1-b58bc4d7c4ef None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:41.875992 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b14b3c-0c75-41f9-8be1-b58bc4d7c4ef None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:41.958493 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d100e4c-adff-4368-bd61-08aa787adcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:41.959428 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d100e4c-adff-4368-bd61-08aa787adcb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:41.959831 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d100e4c-adff-4368-bd61-08aa787adcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:41.960294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d100e4c-adff-4368-bd61-08aa787adcb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:41.966188 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d100e4c-adff-4368-bd61-08aa787adcb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:41.966188 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d100e4c-adff-4368-bd61-08aa787adcb7 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:41.990360 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1242/2463] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:44:41 2022] POST /identity/v3/users => generated 294 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:44:41.995159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936c2ff-949e-4589-8792-fa5a43fd7d8f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:42.006765 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936c2ff-949e-4589-8792-fa5a43fd7d8f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:42.011929 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237f01c-d547-46cf-9749-a746d16845f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:42.012712 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237f01c-d547-46cf-9749-a746d16845f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:42.013047 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237f01c-d547-46cf-9749-a746d16845f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:42.013270 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237f01c-d547-46cf-9749-a746d16845f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:42.013989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3237f01c-d547-46cf-9749-a746d16845f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:42.015683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3237f01c-d547-46cf-9749-a746d16845f7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:42.028356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1222/2464] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:44:41 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:44:42.033966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d100e4c-adff-4368-bd61-08aa787adcb7 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:42.054019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d100e4c-adff-4368-bd61-08aa787adcb7 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:42.062996 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f55f74-7052-4d95-ab78-7d89a8281663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:42.064221 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f55f74-7052-4d95-ab78-7d89a8281663 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:42.064717 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f55f74-7052-4d95-ab78-7d89a8281663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:42.065207 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f55f74-7052-4d95-ab78-7d89a8281663 admin admin] PATH_INFO: `/v3/projects/e7c310eb73944c33ba13d12dc6d33732/users/5fa54b1103a64a25a0be0bf03c2f1dcc/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:42.088858 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f55f74-7052-4d95-ab78-7d89a8281663 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c310eb73944c33ba13d12dc6d33732, user_id=5fa54b1103a64a25a0be0bf03c2f1dcc, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:42.094420 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f55f74-7052-4d95-ab78-7d89a8281663 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:42.119815 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1243/2465] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:44:42 2022] PUT /identity/v3/projects/e7c310eb73944c33ba13d12dc6d33732/users/5fa54b1103a64a25a0be0bf03c2f1dcc/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:44:42.124540 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237f01c-d547-46cf-9749-a746d16845f7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:42.136230 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237f01c-d547-46cf-9749-a746d16845f7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:42.186276 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:44:42.199067 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef407b-7464-4f3b-a3b9-846f21368fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:42.200312 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef407b-7464-4f3b-a3b9-846f21368fcc admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:42.200785 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef407b-7464-4f3b-a3b9-846f21368fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:42.200785 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef407b-7464-4f3b-a3b9-846f21368fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:42.201585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef407b-7464-4f3b-a3b9-846f21368fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:42.203219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef407b-7464-4f3b-a3b9-846f21368fcc admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:42.214436 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1223/2466] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:44:42 2022] GET /identity/v3/roles => generated 1427 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:44:42.219288 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f55f74-7052-4d95-ab78-7d89a8281663 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:42.231061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f55f74-7052-4d95-ab78-7d89a8281663 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:42.235932 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1360f6d-3b9a-4efe-9cff-60f2ba0a653c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:42.236643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1360f6d-3b9a-4efe-9cff-60f2ba0a653c admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:42.236923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1360f6d-3b9a-4efe-9cff-60f2ba0a653c admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:42.237203 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1360f6d-3b9a-4efe-9cff-60f2ba0a653c admin admin] PATH_INFO: `/v3/projects/e7c310eb73944c33ba13d12dc6d33732/users/5fa54b1103a64a25a0be0bf03c2f1dcc/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:42.239253 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1360f6d-3b9a-4efe-9cff-60f2ba0a653c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c310eb73944c33ba13d12dc6d33732, user_id=5fa54b1103a64a25a0be0bf03c2f1dcc, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:42.242086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1360f6d-3b9a-4efe-9cff-60f2ba0a653c admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:42.253269 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1244/2467] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:44:42 2022] PUT /identity/v3/projects/e7c310eb73944c33ba13d12dc6d33732/users/5fa54b1103a64a25a0be0bf03c2f1dcc/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:44:42.265564 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfb864-ed3d-4539-8ff9-4022da6896e5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:42.265958 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfb864-ed3d-4539-8ff9-4022da6896e5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:42.266235 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfb864-ed3d-4539-8ff9-4022da6896e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:42.296592 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-54bfb864-ed3d-4539-8ff9-4022da6896e5 None None] MFA Rules not processed for user `5fa54b1103a64a25a0be0bf03c2f1dcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:44:42.297855 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-54bfb864-ed3d-4539-8ff9-4022da6896e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:42.369634 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1224/2468] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:44:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:44:42.381924 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1360f6d-3b9a-4efe-9cff-60f2ba0a653c admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:42.383799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c1360f6d-3b9a-4efe-9cff-60f2ba0a653c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:42.408273 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:44:42.447748 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1360f6d-3b9a-4efe-9cff-60f2ba0a653c admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:42.452219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2653a0b1-8257-4bd7-aaf7-a1a56a9da140 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:42.452931 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653a0b1-8257-4bd7-aaf7-a1a56a9da140 service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:42.453288 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653a0b1-8257-4bd7-aaf7-a1a56a9da140 service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:42.453615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653a0b1-8257-4bd7-aaf7-a1a56a9da140 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:42.464637 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2653a0b1-8257-4bd7-aaf7-a1a56a9da140 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:42.466008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2653a0b1-8257-4bd7-aaf7-a1a56a9da140 service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:42.477675 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1245/2469] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:44:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:44:42.562688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bfb864-ed3d-4539-8ff9-4022da6896e5 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:42.574884 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bfb864-ed3d-4539-8ff9-4022da6896e5 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:42.579400 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b688b5-799a-47aa-9170-b6a805d03cc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:42.580306 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b688b5-799a-47aa-9170-b6a805d03cc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:42.580589 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b688b5-799a-47aa-9170-b6a805d03cc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:42.580935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b688b5-799a-47aa-9170-b6a805d03cc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:42.651577 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b688b5-799a-47aa-9170-b6a805d03cc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:42.653183 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b688b5-799a-47aa-9170-b6a805d03cc4 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:42.664810 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1225/2470] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:44:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:44:44.052056 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e77d5f-bf86-4868-8350-8cdfb1a1eb77 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:44.052819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e77d5f-bf86-4868-8350-8cdfb1a1eb77 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:44.052819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e77d5f-bf86-4868-8350-8cdfb1a1eb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:44.114704 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-00e77d5f-bf86-4868-8350-8cdfb1a1eb77 None None] MFA Rules not processed for user `efe6e59228bd49bfbff25a11eb682426`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:44:44.116204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-00e77d5f-bf86-4868-8350-8cdfb1a1eb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:44.224999 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1246/2471] 10.222.0.121 () {58 vars in 1014 bytes} [Thu Aug 4 10:44:44 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:44:44.278646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b688b5-799a-47aa-9170-b6a805d03cc4 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:44.280331 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-15b688b5-799a-47aa-9170-b6a805d03cc4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:44.396393 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:44:44.413959 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b688b5-799a-47aa-9170-b6a805d03cc4 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:44.420616 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad723bfc-417e-4555-ac0c-e038808170cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e16e26784601425c9883acd91f65becc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:44.421288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad723bfc-417e-4555-ac0c-e038808170cf service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:44.421575 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad723bfc-417e-4555-ac0c-e038808170cf service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:44.421852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad723bfc-417e-4555-ac0c-e038808170cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:44.432865 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad723bfc-417e-4555-ac0c-e038808170cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:44.434571 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad723bfc-417e-4555-ac0c-e038808170cf service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:44.444338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1226/2472] 10.222.0.121 () {60 vars in 1389 bytes} [Thu Aug 4 10:44:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:44:54.210501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=57 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:44:54.440687 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=8 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:44:58.950916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e77d5f-bf86-4868-8350-8cdfb1a1eb77 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:58.952647 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-00e77d5f-bf86-4868-8350-8cdfb1a1eb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:59.020007 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e77d5f-bf86-4868-8350-8cdfb1a1eb77 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:59.024990 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0a963-721c-49ec-a053-b66ccb5305c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:59.026005 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0a963-721c-49ec-a053-b66ccb5305c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:59.026440 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0a963-721c-49ec-a053-b66ccb5305c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:59.026440 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0a963-721c-49ec-a053-b66ccb5305c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:59.028348 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-5ee0a963-721c-49ec-a053-b66ccb5305c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:59.039811 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0a963-721c-49ec-a053-b66ccb5305c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:59.043820 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0a963-721c-49ec-a053-b66ccb5305c7 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:59.057690 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1247/2473] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:44:58 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:44:59.184480 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0ecb2-1ea0-4d94-ac9e-d9963118716f None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:59.185119 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0ecb2-1ea0-4d94-ac9e-d9963118716f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:59.185119 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0ecb2-1ea0-4d94-ac9e-d9963118716f None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:59.186415 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1227/2474] 10.222.0.121 () {54 vars in 926 bytes} [Thu Aug 4 10:44:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 04 10:44:59.191779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9930e7-5ff6-48e6-affc-a750d38a1886 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:59.192335 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9930e7-5ff6-48e6-affc-a750d38a1886 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:59.192624 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9930e7-5ff6-48e6-affc-a750d38a1886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:59.235824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-de9930e7-5ff6-48e6-affc-a750d38a1886 None None] MFA Rules not processed for user `0d947aeb749546309a8d4b7938087ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:44:59.237188 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-de9930e7-5ff6-48e6-affc-a750d38a1886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:59.310001 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1248/2475] 10.222.0.121 () {58 vars in 1023 bytes} [Thu Aug 4 10:44:59 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:44:59.321707 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0ecb2-1ea0-4d94-ac9e-d9963118716f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:59.324869 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-07a0ecb2-1ea0-4d94-ac9e-d9963118716f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:59.415189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0ecb2-1ea0-4d94-ac9e-d9963118716f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:59.419076 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f64f55-915a-4d9a-8599-3479b8ed5c70 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b31c86bc9334d2b8acc35a035f90442', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:59.419721 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f64f55-915a-4d9a-8599-3479b8ed5c70 service glance] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:59.420117 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f64f55-915a-4d9a-8599-3479b8ed5c70 service glance] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:59.420307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f64f55-915a-4d9a-8599-3479b8ed5c70 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:59.436910 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f64f55-915a-4d9a-8599-3479b8ed5c70 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:59.438456 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f64f55-915a-4d9a-8599-3479b8ed5c70 service glance] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:59.450353 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1228/2476] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:44:59 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:44:59.666468 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9930e7-5ff6-48e6-affc-a750d38a1886 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:44:59.667961 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-de9930e7-5ff6-48e6-affc-a750d38a1886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:59.739636 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9930e7-5ff6-48e6-affc-a750d38a1886 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:44:59.746410 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7a8d5-a40f-4497-8855-400ef3f5da68 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c7cfbe911c4378857848966f4f8d98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:44:59.747065 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7a8d5-a40f-4497-8855-400ef3f5da68 service placement] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:44:59.747359 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7a8d5-a40f-4497-8855-400ef3f5da68 service placement] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:44:59.747643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7a8d5-a40f-4497-8855-400ef3f5da68 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:44:59.749861 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-25c7a8d5-a40f-4497-8855-400ef3f5da68 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:44:59.765089 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7a8d5-a40f-4497-8855-400ef3f5da68 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:44:59.767486 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7a8d5-a40f-4497-8855-400ef3f5da68 service placement] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:44:59.783830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1249/2477] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:44:59 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:45:00.726345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f64f55-915a-4d9a-8599-3479b8ed5c70 service glance] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:00.728091 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-11f64f55-915a-4d9a-8599-3479b8ed5c70 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:00.741378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f64f55-915a-4d9a-8599-3479b8ed5c70 service glance] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:00.745206 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa4e78-1334-4876-a05b-e2464e7fdc14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:00.745820 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa4e78-1334-4876-a05b-e2464e7fdc14 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:00.746141 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa4e78-1334-4876-a05b-e2464e7fdc14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:00.746418 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa4e78-1334-4876-a05b-e2464e7fdc14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:00.748272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-01fa4e78-1334-4876-a05b-e2464e7fdc14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:00.829130 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa4e78-1334-4876-a05b-e2464e7fdc14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:00.830630 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa4e78-1334-4876-a05b-e2464e7fdc14 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:00.840970 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1229/2478] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:45:00 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:45:02.491158 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1d999-068d-47f7-aac0-4c1b9296b0a4 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:02.491731 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1d999-068d-47f7-aac0-4c1b9296b0a4 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:02.491731 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1d999-068d-47f7-aac0-4c1b9296b0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:02.513370 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-92a1d999-068d-47f7-aac0-4c1b9296b0a4 None None] MFA Rules not processed for user `efe6e59228bd49bfbff25a11eb682426`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:02.514521 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-92a1d999-068d-47f7-aac0-4c1b9296b0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:02.522474 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1250/2479] 10.222.0.121 () {58 vars in 1014 bytes} [Thu Aug 4 10:45:02 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:02.535998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa4e78-1334-4876-a05b-e2464e7fdc14 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:02.548181 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa4e78-1334-4876-a05b-e2464e7fdc14 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:02.553861 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c9eb5-9f0e-43cd-8e08-9951aa233673 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e16e26784601425c9883acd91f65becc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:02.554573 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c9eb5-9f0e-43cd-8e08-9951aa233673 service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:02.557157 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c9eb5-9f0e-43cd-8e08-9951aa233673 service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:02.559626 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c9eb5-9f0e-43cd-8e08-9951aa233673 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:02.569811 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c9eb5-9f0e-43cd-8e08-9951aa233673 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:02.571507 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c9eb5-9f0e-43cd-8e08-9951aa233673 service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:02.580355 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1230/2480] 10.222.0.121 () {60 vars in 1389 bytes} [Thu Aug 4 10:45:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:45:05.357426 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1d999-068d-47f7-aac0-4c1b9296b0a4 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:05.360806 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-92a1d999-068d-47f7-aac0-4c1b9296b0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:05.377272 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:05.382692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1d999-068d-47f7-aac0-4c1b9296b0a4 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:05.389892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb08beac-9280-49b0-b9e9-b209d9a76832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:05.391096 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08beac-9280-49b0-b9e9-b209d9a76832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:05.391759 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08beac-9280-49b0-b9e9-b209d9a76832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:05.392465 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08beac-9280-49b0-b9e9-b209d9a76832 admin admin] PATH_INFO: `/v3/users/41a38b5af1654f44bc6cb5832c2e8e87` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:05.394287 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb08beac-9280-49b0-b9e9-b209d9a76832 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41a38b5af1654f44bc6cb5832c2e8e87)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:05.397195 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb08beac-9280-49b0-b9e9-b209d9a76832 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:05.527684 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-cb08beac-9280-49b0-b9e9-b209d9a76832 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41a38b5af1654f44bc6cb5832c2e8e87'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:05.528910 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-cb08beac-9280-49b0-b9e9-b209d9a76832 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41a38b5af1654f44bc6cb5832c2e8e87'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:05.551182 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-cb08beac-9280-49b0-b9e9-b209d9a76832 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41a38b5af1654f44bc6cb5832c2e8e87'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:05.568421 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-cb08beac-9280-49b0-b9e9-b209d9a76832 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41a38b5af1654f44bc6cb5832c2e8e87'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:05.592577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-cb08beac-9280-49b0-b9e9-b209d9a76832 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41a38b5af1654f44bc6cb5832c2e8e87'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:05.601233 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1251/2481] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:45:05 2022] DELETE /identity/v3/users/41a38b5af1654f44bc6cb5832c2e8e87 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:05.882171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c9eb5-9f0e-43cd-8e08-9951aa233673 service swift] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:05.905936 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c9eb5-9f0e-43cd-8e08-9951aa233673 service swift] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:06.027465 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71155a-0f2d-41b5-ba43-99ced87da66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:06.028166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71155a-0f2d-41b5-ba43-99ced87da66c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:06.028297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71155a-0f2d-41b5-ba43-99ced87da66c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:06.028644 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71155a-0f2d-41b5-ba43-99ced87da66c admin admin] PATH_INFO: `/v3/projects/051b6a3defca48d7b4f8bb6705f89f20` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:06.029916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c71155a-0f2d-41b5-ba43-99ced87da66c admin admin] RBAC: Authorizing `identity:delete_project(project_id=051b6a3defca48d7b4f8bb6705f89f20)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:06.031998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c71155a-0f2d-41b5-ba43-99ced87da66c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:06.041048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:06.100996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7c71155a-0f2d-41b5-ba43-99ced87da66c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '051b6a3defca48d7b4f8bb6705f89f20', 'request_id': 'req-7c71155a-0f2d-41b5-ba43-99ced87da66c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:06.112346 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7c71155a-0f2d-41b5-ba43-99ced87da66c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '051b6a3defca48d7b4f8bb6705f89f20', 'request_id': 'req-7c71155a-0f2d-41b5-ba43-99ced87da66c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:06.119783 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7c71155a-0f2d-41b5-ba43-99ced87da66c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '051b6a3defca48d7b4f8bb6705f89f20', 'request_id': 'req-7c71155a-0f2d-41b5-ba43-99ced87da66c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:06.138163 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7c71155a-0f2d-41b5-ba43-99ced87da66c admin admin] The token cache is being invalidate because project 051b6a3defca48d7b4f8bb6705f89f20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:45:06.138479 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7c71155a-0f2d-41b5-ba43-99ced87da66c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:06.139189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7c71155a-0f2d-41b5-ba43-99ced87da66c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:06.140791 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1231/2482] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:45:05 2022] DELETE /identity/v3/projects/051b6a3defca48d7b4f8bb6705f89f20 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:06.146801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e60f429-5910-41af-9400-c8e1a1877b99 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:06.147093 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e60f429-5910-41af-9400-c8e1a1877b99 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:06.147378 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e60f429-5910-41af-9400-c8e1a1877b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:06.182815 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-9e60f429-5910-41af-9400-c8e1a1877b99 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:06.184129 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-9e60f429-5910-41af-9400-c8e1a1877b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:06.250368 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1252/2483] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:06.260160 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008236f-9a45-4c6c-8b94-15ae93a08282 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:06.260977 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008236f-9a45-4c6c-8b94-15ae93a08282 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:06.261637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008236f-9a45-4c6c-8b94-15ae93a08282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:06.289590 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a008236f-9a45-4c6c-8b94-15ae93a08282 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:06.291044 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a008236f-9a45-4c6c-8b94-15ae93a08282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:06.300260 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1232/2484] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:06.306023 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e60f429-5910-41af-9400-c8e1a1877b99 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:06.317985 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e60f429-5910-41af-9400-c8e1a1877b99 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:06.323203 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbadf67-3bee-44d9-a5e5-bfba06e76787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:06.323924 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbadf67-3bee-44d9-a5e5-bfba06e76787 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:06.324237 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbadf67-3bee-44d9-a5e5-bfba06e76787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:06.324515 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbadf67-3bee-44d9-a5e5-bfba06e76787 admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:06.325242 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbadf67-3bee-44d9-a5e5-bfba06e76787 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:45:06.325814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbadf67-3bee-44d9-a5e5-bfba06e76787 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:06.327480 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbadf67-3bee-44d9-a5e5-bfba06e76787 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:06.342381 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1253/2485] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:45:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:45:06.347292 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a008236f-9a45-4c6c-8b94-15ae93a08282 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:06.357399 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a008236f-9a45-4c6c-8b94-15ae93a08282 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:06.362248 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010b4e7-0ad6-4146-b72f-7f71d3368f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:06.362904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010b4e7-0ad6-4146-b72f-7f71d3368f39 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:06.363194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010b4e7-0ad6-4146-b72f-7f71d3368f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:06.363476 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010b4e7-0ad6-4146-b72f-7f71d3368f39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:06.364234 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7010b4e7-0ad6-4146-b72f-7f71d3368f39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:06.366237 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7010b4e7-0ad6-4146-b72f-7f71d3368f39 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:06.408233 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1233/2486] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:45:06 2022] POST /identity/v3/projects => generated 358 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:06.412956 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbadf67-3bee-44d9-a5e5-bfba06e76787 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:06.427564 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbadf67-3bee-44d9-a5e5-bfba06e76787 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:06.510052 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b684f-5e6a-4693-9e11-cb364d002bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:06.510723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b684f-5e6a-4693-9e11-cb364d002bb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:06.511013 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b684f-5e6a-4693-9e11-cb364d002bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:06.511381 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b684f-5e6a-4693-9e11-cb364d002bb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:06.512033 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b684f-5e6a-4693-9e11-cb364d002bb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:06.514160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b684f-5e6a-4693-9e11-cb364d002bb7 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:06.525606 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:06.548903 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1254/2487] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:45:06 2022] POST /identity/v3/users => generated 280 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:06.550243 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010b4e7-0ad6-4146-b72f-7f71d3368f39 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:06.566868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010b4e7-0ad6-4146-b72f-7f71d3368f39 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:06.582032 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5c336-9c0f-4e7b-87df-048fa6033284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:06.582678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5c336-9c0f-4e7b-87df-048fa6033284 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:06.582999 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5c336-9c0f-4e7b-87df-048fa6033284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:06.584153 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5c336-9c0f-4e7b-87df-048fa6033284 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:06.584153 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de5c336-9c0f-4e7b-87df-048fa6033284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:06.586691 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de5c336-9c0f-4e7b-87df-048fa6033284 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:06.601421 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1234/2488] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:06 2022] GET /identity/v3/roles => generated 1427 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:06.614765 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b684f-5e6a-4693-9e11-cb364d002bb7 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:06.626301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b684f-5e6a-4693-9e11-cb364d002bb7 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:06.632330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8071d-517d-46ed-8c40-ddbfb0c0b23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:06.633053 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8071d-517d-46ed-8c40-ddbfb0c0b23e admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:06.633533 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8071d-517d-46ed-8c40-ddbfb0c0b23e admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:06.634251 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8071d-517d-46ed-8c40-ddbfb0c0b23e admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:06.635011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b8071d-517d-46ed-8c40-ddbfb0c0b23e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:06.636708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b8071d-517d-46ed-8c40-ddbfb0c0b23e admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:06.662722 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1255/2489] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:06 2022] GET /identity/v3/roles => generated 1427 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:06.667615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5c336-9c0f-4e7b-87df-048fa6033284 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:06.681023 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5c336-9c0f-4e7b-87df-048fa6033284 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:06.693389 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab94f022-84db-4f78-baea-e2aa5b8cf7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:06.695135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94f022-84db-4f78-baea-e2aa5b8cf7d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:06.696003 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94f022-84db-4f78-baea-e2aa5b8cf7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:06.696796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94f022-84db-4f78-baea-e2aa5b8cf7d7 admin admin] PATH_INFO: `/v3/projects/4975a0490ffa46c3b000335f68520169/users/24f81f4f1d894879bfe2c417861be450/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:06.733744 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab94f022-84db-4f78-baea-e2aa5b8cf7d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4975a0490ffa46c3b000335f68520169, user_id=24f81f4f1d894879bfe2c417861be450, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:06.737256 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab94f022-84db-4f78-baea-e2aa5b8cf7d7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:06.771006 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1235/2490] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:06 2022] PUT /identity/v3/projects/4975a0490ffa46c3b000335f68520169/users/24f81f4f1d894879bfe2c417861be450/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:06.790815 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842115d-3ca2-46ac-8a1d-3a1ab79dc1f7 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:06.791318 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842115d-3ca2-46ac-8a1d-3a1ab79dc1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:06.791794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842115d-3ca2-46ac-8a1d-3a1ab79dc1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:06.854906 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-6842115d-3ca2-46ac-8a1d-3a1ab79dc1f7 None None] MFA Rules not processed for user `24f81f4f1d894879bfe2c417861be450`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:06.856387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6842115d-3ca2-46ac-8a1d-3a1ab79dc1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:06.975350 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1256/2491] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:06.980323 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab94f022-84db-4f78-baea-e2aa5b8cf7d7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:07.104990 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab94f022-84db-4f78-baea-e2aa5b8cf7d7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:07.111719 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00365f9-e8a5-481c-aea6-85add473dd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:07.112893 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00365f9-e8a5-481c-aea6-85add473dd53 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:07.113287 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00365f9-e8a5-481c-aea6-85add473dd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:07.113742 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00365f9-e8a5-481c-aea6-85add473dd53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:07.114768 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00365f9-e8a5-481c-aea6-85add473dd53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:07.117710 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00365f9-e8a5-481c-aea6-85add473dd53 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:07.174033 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1236/2492] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:45:06 2022] POST /identity/v3/projects => generated 356 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:07.192481 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842115d-3ca2-46ac-8a1d-3a1ab79dc1f7 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:07.217671 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842115d-3ca2-46ac-8a1d-3a1ab79dc1f7 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:07.356426 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb69d9-84a7-4276-8139-0e014c925358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:07.357128 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb69d9-84a7-4276-8139-0e014c925358 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:07.357380 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb69d9-84a7-4276-8139-0e014c925358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:07.357672 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb69d9-84a7-4276-8139-0e014c925358 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:07.358411 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb69d9-84a7-4276-8139-0e014c925358 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:07.360383 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb69d9-84a7-4276-8139-0e014c925358 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:07.391041 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1257/2493] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:45:07 2022] POST /identity/v3/users => generated 285 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:07.395463 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00365f9-e8a5-481c-aea6-85add473dd53 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:07.409631 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00365f9-e8a5-481c-aea6-85add473dd53 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:07.415089 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfbe1e-7284-44d2-8842-34039272ca6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:07.415756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfbe1e-7284-44d2-8842-34039272ca6d admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:07.416093 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfbe1e-7284-44d2-8842-34039272ca6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:07.416462 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfbe1e-7284-44d2-8842-34039272ca6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:07.417195 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccfbe1e-7284-44d2-8842-34039272ca6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:07.419538 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccfbe1e-7284-44d2-8842-34039272ca6d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:07.433532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1237/2494] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:07 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:07.438323 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb69d9-84a7-4276-8139-0e014c925358 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:07.453411 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb69d9-84a7-4276-8139-0e014c925358 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:07.458575 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2625c-87af-41ac-af3f-dfd2019f599d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:07.459424 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2625c-87af-41ac-af3f-dfd2019f599d admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:07.459784 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2625c-87af-41ac-af3f-dfd2019f599d admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:07.460332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2625c-87af-41ac-af3f-dfd2019f599d admin admin] PATH_INFO: `/v3/projects/897d897edc684c78b7cef450c49330e6/users/c67333b8719847dfacbabd48aeedbfab/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:07.479939 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf2625c-87af-41ac-af3f-dfd2019f599d admin admin] RBAC: Authorizing `identity:create_grant(project_id=897d897edc684c78b7cef450c49330e6, user_id=c67333b8719847dfacbabd48aeedbfab, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:07.483188 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf2625c-87af-41ac-af3f-dfd2019f599d admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:07.505463 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1258/2495] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:07 2022] PUT /identity/v3/projects/897d897edc684c78b7cef450c49330e6/users/c67333b8719847dfacbabd48aeedbfab/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:07.510352 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfbe1e-7284-44d2-8842-34039272ca6d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:07.517340 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:07.522212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfbe1e-7284-44d2-8842-34039272ca6d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:07.605909 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c66a58-eb8b-46e6-abfd-d068762cbde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:07.606559 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c66a58-eb8b-46e6-abfd-d068762cbde4 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:07.606848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c66a58-eb8b-46e6-abfd-d068762cbde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:07.607131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c66a58-eb8b-46e6-abfd-d068762cbde4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:07.607852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c66a58-eb8b-46e6-abfd-d068762cbde4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:07.609525 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c66a58-eb8b-46e6-abfd-d068762cbde4 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:07.622147 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1238/2496] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:07 2022] GET /identity/v3/roles => generated 1427 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:07.626894 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2625c-87af-41ac-af3f-dfd2019f599d admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:07.639549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2625c-87af-41ac-af3f-dfd2019f599d admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:07.644461 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf38c4a-bfb1-4511-9051-d3592e232673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:07.645206 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf38c4a-bfb1-4511-9051-d3592e232673 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:07.645499 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf38c4a-bfb1-4511-9051-d3592e232673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:07.645782 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf38c4a-bfb1-4511-9051-d3592e232673 admin admin] PATH_INFO: `/v3/projects/897d897edc684c78b7cef450c49330e6/users/c67333b8719847dfacbabd48aeedbfab/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:07.648749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf38c4a-bfb1-4511-9051-d3592e232673 admin admin] RBAC: Authorizing `identity:create_grant(project_id=897d897edc684c78b7cef450c49330e6, user_id=c67333b8719847dfacbabd48aeedbfab, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:07.651493 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf38c4a-bfb1-4511-9051-d3592e232673 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:07.663712 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1259/2497] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:07 2022] PUT /identity/v3/projects/897d897edc684c78b7cef450c49330e6/users/c67333b8719847dfacbabd48aeedbfab/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:07.676355 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdbd2e-e2b3-441a-b4da-ab6b0475426f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:07.676717 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdbd2e-e2b3-441a-b4da-ab6b0475426f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:07.676919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdbd2e-e2b3-441a-b4da-ab6b0475426f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:07.706289 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f6cdbd2e-e2b3-441a-b4da-ab6b0475426f None None] MFA Rules not processed for user `c67333b8719847dfacbabd48aeedbfab`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:07.707557 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f6cdbd2e-e2b3-441a-b4da-ab6b0475426f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:07.787504 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1239/2498] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:07 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:07.800075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf38c4a-bfb1-4511-9051-d3592e232673 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:07.802274 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-dcf38c4a-bfb1-4511-9051-d3592e232673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:07.833625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:07.868585 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf38c4a-bfb1-4511-9051-d3592e232673 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:07.872546 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c88ef-98c0-45c3-a6ae-8540c1e3c11e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:07.873191 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c88ef-98c0-45c3-a6ae-8540c1e3c11e service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:07.873475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c88ef-98c0-45c3-a6ae-8540c1e3c11e service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:07.873749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c88ef-98c0-45c3-a6ae-8540c1e3c11e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:07.883190 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c88ef-98c0-45c3-a6ae-8540c1e3c11e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:07.884742 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c88ef-98c0-45c3-a6ae-8540c1e3c11e service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:07.895995 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1260/2499] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:45:07 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:45:07.983614 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cdbd2e-e2b3-441a-b4da-ab6b0475426f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:07.995792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cdbd2e-e2b3-441a-b4da-ab6b0475426f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:08.000391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13a659-6db1-46a0-918f-3203f755511f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:08.001475 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13a659-6db1-46a0-918f-3203f755511f service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:08.001857 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13a659-6db1-46a0-918f-3203f755511f service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:08.002220 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13a659-6db1-46a0-918f-3203f755511f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:08.084381 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13a659-6db1-46a0-918f-3203f755511f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:08.086673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13a659-6db1-46a0-918f-3203f755511f service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:08.100355 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1240/2500] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:45:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:45:17.890550 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=15 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:18.095150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:33.065504 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c88ef-98c0-45c3-a6ae-8540c1e3c11e service cinder] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:33.141870 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c88ef-98c0-45c3-a6ae-8540c1e3c11e service cinder] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:33.146662 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903d2cd-046f-4ed4-8875-91feb44e1e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:33.147344 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903d2cd-046f-4ed4-8875-91feb44e1e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:33.147642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903d2cd-046f-4ed4-8875-91feb44e1e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:33.147938 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903d2cd-046f-4ed4-8875-91feb44e1e85 admin admin] PATH_INFO: `/v3/users/24f81f4f1d894879bfe2c417861be450` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:33.149476 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903d2cd-046f-4ed4-8875-91feb44e1e85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24f81f4f1d894879bfe2c417861be450)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:33.151525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903d2cd-046f-4ed4-8875-91feb44e1e85 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:33.208718 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-d903d2cd-046f-4ed4-8875-91feb44e1e85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24f81f4f1d894879bfe2c417861be450'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:33.209497 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-d903d2cd-046f-4ed4-8875-91feb44e1e85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24f81f4f1d894879bfe2c417861be450'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:33.225255 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-d903d2cd-046f-4ed4-8875-91feb44e1e85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24f81f4f1d894879bfe2c417861be450'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:33.232337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-d903d2cd-046f-4ed4-8875-91feb44e1e85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24f81f4f1d894879bfe2c417861be450'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:33.241510 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-d903d2cd-046f-4ed4-8875-91feb44e1e85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24f81f4f1d894879bfe2c417861be450'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:33.248090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1261/2501] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:45:33 2022] DELETE /identity/v3/users/24f81f4f1d894879bfe2c417861be450 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:33.253561 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb4bb3-85c7-4b8d-ba79-ba5c18d86472 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:33.254112 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb4bb3-85c7-4b8d-ba79-ba5c18d86472 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:33.254216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb4bb3-85c7-4b8d-ba79-ba5c18d86472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:33.277869 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-55bb4bb3-85c7-4b8d-ba79-ba5c18d86472 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:33.279869 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-55bb4bb3-85c7-4b8d-ba79-ba5c18d86472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:33.347024 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1241/2502] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:45:33.357707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903d2cd-046f-4ed4-8875-91feb44e1e85 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:33.359451 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d903d2cd-046f-4ed4-8875-91feb44e1e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:33.421485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903d2cd-046f-4ed4-8875-91feb44e1e85 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:33.426503 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cfb609-dea8-49a3-b310-c1c84eead395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:33.427134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfb609-dea8-49a3-b310-c1c84eead395 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:33.427452 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfb609-dea8-49a3-b310-c1c84eead395 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:33.427731 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfb609-dea8-49a3-b310-c1c84eead395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:33.439209 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cfb609-dea8-49a3-b310-c1c84eead395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:33.440725 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cfb609-dea8-49a3-b310-c1c84eead395 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:33.450878 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1262/2503] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:45:33 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:45:33.750935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb4bb3-85c7-4b8d-ba79-ba5c18d86472 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:33.753592 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-55bb4bb3-85c7-4b8d-ba79-ba5c18d86472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:33.769374 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb4bb3-85c7-4b8d-ba79-ba5c18d86472 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:33.781069 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477346b-a444-4c83-a933-a8d7b7abd79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:33.781964 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477346b-a444-4c83-a933-a8d7b7abd79b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:33.782511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477346b-a444-4c83-a933-a8d7b7abd79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:33.783062 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477346b-a444-4c83-a933-a8d7b7abd79b admin admin] PATH_INFO: `/v3/projects/4975a0490ffa46c3b000335f68520169` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:33.784783 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4477346b-a444-4c83-a933-a8d7b7abd79b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4975a0490ffa46c3b000335f68520169)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:33.787905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4477346b-a444-4c83-a933-a8d7b7abd79b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:33.840852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4477346b-a444-4c83-a933-a8d7b7abd79b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4975a0490ffa46c3b000335f68520169', 'request_id': 'req-4477346b-a444-4c83-a933-a8d7b7abd79b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:33.847515 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4477346b-a444-4c83-a933-a8d7b7abd79b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4975a0490ffa46c3b000335f68520169', 'request_id': 'req-4477346b-a444-4c83-a933-a8d7b7abd79b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:33.853183 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4477346b-a444-4c83-a933-a8d7b7abd79b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4975a0490ffa46c3b000335f68520169', 'request_id': 'req-4477346b-a444-4c83-a933-a8d7b7abd79b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:33.867798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4477346b-a444-4c83-a933-a8d7b7abd79b admin admin] The token cache is being invalidate because project 4975a0490ffa46c3b000335f68520169 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:45:33.868119 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4477346b-a444-4c83-a933-a8d7b7abd79b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:33.868890 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4477346b-a444-4c83-a933-a8d7b7abd79b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:33.870695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1242/2504] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:45:33 2022] DELETE /identity/v3/projects/4975a0490ffa46c3b000335f68520169 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:33.876873 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cfb609-dea8-49a3-b310-c1c84eead395 service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:33.890612 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cfb609-dea8-49a3-b310-c1c84eead395 service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:33.911246 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:33.953116 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae802f0b-1f47-4ece-bbb5-e5f753ea3440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:33.953678 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae802f0b-1f47-4ece-bbb5-e5f753ea3440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:33.954027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae802f0b-1f47-4ece-bbb5-e5f753ea3440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:33.954222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae802f0b-1f47-4ece-bbb5-e5f753ea3440 admin admin] PATH_INFO: `/v3/users/c67333b8719847dfacbabd48aeedbfab` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:33.955587 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae802f0b-1f47-4ece-bbb5-e5f753ea3440 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c67333b8719847dfacbabd48aeedbfab)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:33.957609 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae802f0b-1f47-4ece-bbb5-e5f753ea3440 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:34.012921 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-ae802f0b-1f47-4ece-bbb5-e5f753ea3440 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c67333b8719847dfacbabd48aeedbfab'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:34.013596 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-ae802f0b-1f47-4ece-bbb5-e5f753ea3440 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c67333b8719847dfacbabd48aeedbfab'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:34.025859 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-ae802f0b-1f47-4ece-bbb5-e5f753ea3440 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c67333b8719847dfacbabd48aeedbfab'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:34.033057 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-ae802f0b-1f47-4ece-bbb5-e5f753ea3440 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c67333b8719847dfacbabd48aeedbfab'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:34.044140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-ae802f0b-1f47-4ece-bbb5-e5f753ea3440 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c67333b8719847dfacbabd48aeedbfab'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:34.052031 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1263/2505] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:45:33 2022] DELETE /identity/v3/users/c67333b8719847dfacbabd48aeedbfab => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:34.368643 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477346b-a444-4c83-a933-a8d7b7abd79b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:34.368643 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4477346b-a444-4c83-a933-a8d7b7abd79b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:34.393572 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477346b-a444-4c83-a933-a8d7b7abd79b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:34.455569 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2357df3-033f-44e6-b602-80788ebc721a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:34.456164 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2357df3-033f-44e6-b602-80788ebc721a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:34.456454 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2357df3-033f-44e6-b602-80788ebc721a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:34.456736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2357df3-033f-44e6-b602-80788ebc721a admin admin] PATH_INFO: `/v3/projects/897d897edc684c78b7cef450c49330e6` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:34.457932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2357df3-033f-44e6-b602-80788ebc721a admin admin] RBAC: Authorizing `identity:delete_project(project_id=897d897edc684c78b7cef450c49330e6)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:34.459885 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2357df3-033f-44e6-b602-80788ebc721a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:34.496915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:34.522933 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d2357df3-033f-44e6-b602-80788ebc721a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '897d897edc684c78b7cef450c49330e6', 'request_id': 'req-d2357df3-033f-44e6-b602-80788ebc721a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:34.530447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d2357df3-033f-44e6-b602-80788ebc721a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '897d897edc684c78b7cef450c49330e6', 'request_id': 'req-d2357df3-033f-44e6-b602-80788ebc721a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:34.536623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d2357df3-033f-44e6-b602-80788ebc721a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '897d897edc684c78b7cef450c49330e6', 'request_id': 'req-d2357df3-033f-44e6-b602-80788ebc721a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:34.551129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d2357df3-033f-44e6-b602-80788ebc721a admin admin] The token cache is being invalidate because project 897d897edc684c78b7cef450c49330e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:45:34.551422 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d2357df3-033f-44e6-b602-80788ebc721a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:34.552168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d2357df3-033f-44e6-b602-80788ebc721a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:34.554078 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1243/2506] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:45:34 2022] DELETE /identity/v3/projects/897d897edc684c78b7cef450c49330e6 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:34.559859 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a3cde-a7be-4e08-939b-c75fafc6b7dc None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:34.560432 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a3cde-a7be-4e08-939b-c75fafc6b7dc None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:34.560432 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a3cde-a7be-4e08-939b-c75fafc6b7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:34.586559 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-0b3a3cde-a7be-4e08-939b-c75fafc6b7dc None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:34.587359 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-0b3a3cde-a7be-4e08-939b-c75fafc6b7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:34.648427 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1264/2507] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:34 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:34.656989 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26285663-e3cc-4c06-aaae-6a1dd648b7b9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:34.657296 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26285663-e3cc-4c06-aaae-6a1dd648b7b9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:34.657609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26285663-e3cc-4c06-aaae-6a1dd648b7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:34.678601 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-26285663-e3cc-4c06-aaae-6a1dd648b7b9 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:34.679856 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-26285663-e3cc-4c06-aaae-6a1dd648b7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:34.688302 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1244/2508] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:34 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:34.693081 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a3cde-a7be-4e08-939b-c75fafc6b7dc None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:34.704875 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a3cde-a7be-4e08-939b-c75fafc6b7dc None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:34.710456 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f7b6b-6916-41f7-8819-c132f3d1445e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:34.711069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f7b6b-6916-41f7-8819-c132f3d1445e admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:34.711354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f7b6b-6916-41f7-8819-c132f3d1445e admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:34.712036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f7b6b-6916-41f7-8819-c132f3d1445e admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:34.712756 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f7b6b-6916-41f7-8819-c132f3d1445e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:45:34.713068 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f7b6b-6916-41f7-8819-c132f3d1445e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:34.714974 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f7b6b-6916-41f7-8819-c132f3d1445e admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:34.730900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1265/2509] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:45:34 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:45:34.735665 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26285663-e3cc-4c06-aaae-6a1dd648b7b9 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:34.745764 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26285663-e3cc-4c06-aaae-6a1dd648b7b9 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:34.751498 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f91707-54ca-4159-8381-51f7dfe21658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:34.752229 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f91707-54ca-4159-8381-51f7dfe21658 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:34.752522 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f91707-54ca-4159-8381-51f7dfe21658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:34.752799 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f91707-54ca-4159-8381-51f7dfe21658 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:34.753515 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f91707-54ca-4159-8381-51f7dfe21658 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:34.755536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f91707-54ca-4159-8381-51f7dfe21658 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:34.793693 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1245/2510] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:45:34 2022] POST /identity/v3/projects => generated 366 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:34.798432 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f7b6b-6916-41f7-8819-c132f3d1445e admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:34.810674 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f7b6b-6916-41f7-8819-c132f3d1445e admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:34.853024 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:34.890384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c5c67-b6f3-4c63-a25a-6d9a8f663ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:34.891261 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c5c67-b6f3-4c63-a25a-6d9a8f663ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:34.891551 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c5c67-b6f3-4c63-a25a-6d9a8f663ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:34.891833 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c5c67-b6f3-4c63-a25a-6d9a8f663ea6 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:34.892556 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c5c67-b6f3-4c63-a25a-6d9a8f663ea6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:34.894514 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c5c67-b6f3-4c63-a25a-6d9a8f663ea6 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:34.926305 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1266/2511] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:45:34 2022] POST /identity/v3/users => generated 284 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:34.930664 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f91707-54ca-4159-8381-51f7dfe21658 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:34.942928 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f91707-54ca-4159-8381-51f7dfe21658 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:34.947926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987f5a4-865f-40b8-8698-f9d03d6d3b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:34.948566 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987f5a4-865f-40b8-8698-f9d03d6d3b43 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:34.948853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987f5a4-865f-40b8-8698-f9d03d6d3b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:34.949133 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987f5a4-865f-40b8-8698-f9d03d6d3b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:34.949849 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987f5a4-865f-40b8-8698-f9d03d6d3b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:34.951484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987f5a4-865f-40b8-8698-f9d03d6d3b43 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:34.963690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1246/2512] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:34 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:34.967785 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c5c67-b6f3-4c63-a25a-6d9a8f663ea6 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:34.978928 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c5c67-b6f3-4c63-a25a-6d9a8f663ea6 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:34.984937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cf0f60-5d5b-4364-a701-bae70c847ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:34.985587 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf0f60-5d5b-4364-a701-bae70c847ebe admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:34.985876 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf0f60-5d5b-4364-a701-bae70c847ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:34.986156 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf0f60-5d5b-4364-a701-bae70c847ebe admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:34.986875 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cf0f60-5d5b-4364-a701-bae70c847ebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:34.988558 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cf0f60-5d5b-4364-a701-bae70c847ebe admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:35.001839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1267/2513] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:34 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:35.006448 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987f5a4-865f-40b8-8698-f9d03d6d3b43 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:35.019679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987f5a4-865f-40b8-8698-f9d03d6d3b43 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:35.025182 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805501d9-8ac8-456e-bcb2-02846b025f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:35.025906 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805501d9-8ac8-456e-bcb2-02846b025f79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:35.026258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805501d9-8ac8-456e-bcb2-02846b025f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:35.026534 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805501d9-8ac8-456e-bcb2-02846b025f79 admin admin] PATH_INFO: `/v3/projects/480d3681ebe34d38b5504e064206b76d/users/d1ae1d8d8b8d48438c816b4a468cb96a/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:35.044023 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805501d9-8ac8-456e-bcb2-02846b025f79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=480d3681ebe34d38b5504e064206b76d, user_id=d1ae1d8d8b8d48438c816b4a468cb96a, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:35.046695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805501d9-8ac8-456e-bcb2-02846b025f79 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:35.048762 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df7cb3-f7b8-46cf-9c53-d025c617bc44 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:35.049160 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df7cb3-f7b8-46cf-9c53-d025c617bc44 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:35.049448 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df7cb3-f7b8-46cf-9c53-d025c617bc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:35.062449 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1247/2514] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:35 2022] PUT /identity/v3/projects/480d3681ebe34d38b5504e064206b76d/users/d1ae1d8d8b8d48438c816b4a468cb96a/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:35.072181 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-c5df7cb3-f7b8-46cf-9c53-d025c617bc44 None None] MFA Rules not processed for user `efe6e59228bd49bfbff25a11eb682426`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:35.073788 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c5df7cb3-f7b8-46cf-9c53-d025c617bc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:35.074878 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d202de2-8b55-4cef-9b11-9ed99eca7c2c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:35.075239 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d202de2-8b55-4cef-9b11-9ed99eca7c2c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:35.075528 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d202de2-8b55-4cef-9b11-9ed99eca7c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:35.105229 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8d202de2-8b55-4cef-9b11-9ed99eca7c2c None None] MFA Rules not processed for user `d1ae1d8d8b8d48438c816b4a468cb96a`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:35.106498 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8d202de2-8b55-4cef-9b11-9ed99eca7c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:35.135566 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1268/2515] 10.222.0.121 () {58 vars in 972 bytes} [Thu Aug 4 10:45:35 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:35.151769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df7cb3-f7b8-46cf-9c53-d025c617bc44 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:35.153378 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c5df7cb3-f7b8-46cf-9c53-d025c617bc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:35.189676 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1248/2516] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:35.190912 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d202de2-8b55-4cef-9b11-9ed99eca7c2c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:35.233791 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df7cb3-f7b8-46cf-9c53-d025c617bc44 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:35.238921 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:35.239511 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36cb8e-0f57-41fe-aa97-217ada84e074 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e16e26784601425c9883acd91f65becc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:35.240262 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36cb8e-0f57-41fe-aa97-217ada84e074 service swift] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:35.240590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36cb8e-0f57-41fe-aa97-217ada84e074 service swift] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:35.240871 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36cb8e-0f57-41fe-aa97-217ada84e074 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:35.250746 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36cb8e-0f57-41fe-aa97-217ada84e074 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:35.252413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36cb8e-0f57-41fe-aa97-217ada84e074 service swift] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:35.261174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1269/2517] 10.222.0.121 () {60 vars in 1389 bytes} [Thu Aug 4 10:45:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 10:45:35.261691 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d202de2-8b55-4cef-9b11-9ed99eca7c2c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:35.267225 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5e746-c312-4c78-bb51-19fc6cbf5f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:35.268262 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5e746-c312-4c78-bb51-19fc6cbf5f16 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:35.268536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5e746-c312-4c78-bb51-19fc6cbf5f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:35.269357 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5e746-c312-4c78-bb51-19fc6cbf5f16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:35.269563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e5e746-c312-4c78-bb51-19fc6cbf5f16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:35.271869 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e5e746-c312-4c78-bb51-19fc6cbf5f16 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:35.302339 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1249/2518] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:45:35 2022] POST /identity/v3/projects => generated 364 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:35.307238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36cb8e-0f57-41fe-aa97-217ada84e074 service swift] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:35.318578 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36cb8e-0f57-41fe-aa97-217ada84e074 service swift] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:35.362524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:35.382536 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f9a66-46ea-4f9b-b661-ed4ac04f79ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:35.383201 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f9a66-46ea-4f9b-b661-ed4ac04f79ea admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:35.383598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f9a66-46ea-4f9b-b661-ed4ac04f79ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:35.383900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f9a66-46ea-4f9b-b661-ed4ac04f79ea admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:35.384615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f9a66-46ea-4f9b-b661-ed4ac04f79ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:35.386696 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f9a66-46ea-4f9b-b661-ed4ac04f79ea admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:35.420241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1270/2519] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:45:35 2022] POST /identity/v3/users => generated 289 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:35.423114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5e746-c312-4c78-bb51-19fc6cbf5f16 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:35.435325 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5e746-c312-4c78-bb51-19fc6cbf5f16 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:35.440392 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4906e26-9743-48f9-8d71-072f72b74f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:35.441034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4906e26-9743-48f9-8d71-072f72b74f6e admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:35.441327 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4906e26-9743-48f9-8d71-072f72b74f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:35.441620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4906e26-9743-48f9-8d71-072f72b74f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:35.442319 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4906e26-9743-48f9-8d71-072f72b74f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:35.444107 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4906e26-9743-48f9-8d71-072f72b74f6e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:35.456597 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1250/2520] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:35 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:35.460966 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f9a66-46ea-4f9b-b661-ed4ac04f79ea admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:35.472171 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f9a66-46ea-4f9b-b661-ed4ac04f79ea admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:35.477218 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617f775-7161-4cfd-b5c2-66c6c0a96634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:35.477923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617f775-7161-4cfd-b5c2-66c6c0a96634 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:35.478210 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617f775-7161-4cfd-b5c2-66c6c0a96634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:35.478491 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617f775-7161-4cfd-b5c2-66c6c0a96634 admin admin] PATH_INFO: `/v3/projects/5f7292b523bc4d86b94ee4bf1b10f609/users/430cf655dc954ba38edaf0b0a790b4bb/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:35.495508 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4617f775-7161-4cfd-b5c2-66c6c0a96634 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f7292b523bc4d86b94ee4bf1b10f609, user_id=430cf655dc954ba38edaf0b0a790b4bb, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:35.498594 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4617f775-7161-4cfd-b5c2-66c6c0a96634 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:35.513168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1271/2521] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:35 2022] PUT /identity/v3/projects/5f7292b523bc4d86b94ee4bf1b10f609/users/430cf655dc954ba38edaf0b0a790b4bb/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:35.518084 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4906e26-9743-48f9-8d71-072f72b74f6e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:35.529914 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4906e26-9743-48f9-8d71-072f72b74f6e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:35.607412 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed3069-e9e8-41cd-acc0-53233deaef5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:35.608262 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed3069-e9e8-41cd-acc0-53233deaef5e admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:35.608430 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed3069-e9e8-41cd-acc0-53233deaef5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:35.608699 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed3069-e9e8-41cd-acc0-53233deaef5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:35.609467 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed3069-e9e8-41cd-acc0-53233deaef5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:35.611274 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed3069-e9e8-41cd-acc0-53233deaef5e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:35.623478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1251/2522] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:35 2022] GET /identity/v3/roles => generated 1427 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:35.627937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617f775-7161-4cfd-b5c2-66c6c0a96634 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:35.639621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617f775-7161-4cfd-b5c2-66c6c0a96634 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:35.646335 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae0a88-43be-45cf-a475-352d206d5bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:35.647213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae0a88-43be-45cf-a475-352d206d5bfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:35.647784 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae0a88-43be-45cf-a475-352d206d5bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:35.648102 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae0a88-43be-45cf-a475-352d206d5bfc admin admin] PATH_INFO: `/v3/projects/5f7292b523bc4d86b94ee4bf1b10f609/users/430cf655dc954ba38edaf0b0a790b4bb/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:35.655595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae0a88-43be-45cf-a475-352d206d5bfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f7292b523bc4d86b94ee4bf1b10f609, user_id=430cf655dc954ba38edaf0b0a790b4bb, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:35.655595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae0a88-43be-45cf-a475-352d206d5bfc admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:35.667352 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1272/2523] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:35 2022] PUT /identity/v3/projects/5f7292b523bc4d86b94ee4bf1b10f609/users/430cf655dc954ba38edaf0b0a790b4bb/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:35.678326 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ff09c-bb70-4d52-9b24-d61eb3c39c69 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:35.678606 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ff09c-bb70-4d52-9b24-d61eb3c39c69 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:35.678868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ff09c-bb70-4d52-9b24-d61eb3c39c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:35.714998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9c3ff09c-bb70-4d52-9b24-d61eb3c39c69 None None] MFA Rules not processed for user `430cf655dc954ba38edaf0b0a790b4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:35.716298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9c3ff09c-bb70-4d52-9b24-d61eb3c39c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:35.797475 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1252/2524] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:35.812918 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae0a88-43be-45cf-a475-352d206d5bfc admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:35.815139 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-eeae0a88-43be-45cf-a475-352d206d5bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:35.890993 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae0a88-43be-45cf-a475-352d206d5bfc admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:35.895640 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e40e7c-a432-4790-86b6-b57e238dc0bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:35.896442 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e40e7c-a432-4790-86b6-b57e238dc0bb service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:35.896739 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e40e7c-a432-4790-86b6-b57e238dc0bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:35.896884 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e40e7c-a432-4790-86b6-b57e238dc0bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:35.914598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e40e7c-a432-4790-86b6-b57e238dc0bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:35.916497 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e40e7c-a432-4790-86b6-b57e238dc0bb service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:35.928770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1273/2525] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:45:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:45:36.020698 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ff09c-bb70-4d52-9b24-d61eb3c39c69 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:36.022995 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9c3ff09c-bb70-4d52-9b24-d61eb3c39c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:36.024751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e40e7c-a432-4790-86b6-b57e238dc0bb service cinder] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:36.033962 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e40e7c-a432-4790-86b6-b57e238dc0bb service cinder] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:36.037999 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4186ef70-d9a9-47fc-9b88-4a67e772518c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:36.038679 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186ef70-d9a9-47fc-9b88-4a67e772518c service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:36.038973 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186ef70-d9a9-47fc-9b88-4a67e772518c service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:36.039254 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186ef70-d9a9-47fc-9b88-4a67e772518c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:36.071697 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:36.106112 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ff09c-bb70-4d52-9b24-d61eb3c39c69 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:36.107445 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4186ef70-d9a9-47fc-9b88-4a67e772518c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:36.109251 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4186ef70-d9a9-47fc-9b88-4a67e772518c service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:36.111239 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826be7e-3584-4f14-b24c-4cee4bc38a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:36.112012 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826be7e-3584-4f14-b24c-4cee4bc38a32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:36.112375 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826be7e-3584-4f14-b24c-4cee4bc38a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:36.112658 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826be7e-3584-4f14-b24c-4cee4bc38a32 admin admin] PATH_INFO: `/v3/users/94bd01a2aa764b6583fd477015f56c82` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:36.113902 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8826be7e-3584-4f14-b24c-4cee4bc38a32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94bd01a2aa764b6583fd477015f56c82)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:36.116072 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8826be7e-3584-4f14-b24c-4cee4bc38a32 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:36.121372 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1274/2526] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:45:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:45:36.178551 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8826be7e-3584-4f14-b24c-4cee4bc38a32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94bd01a2aa764b6583fd477015f56c82'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:36.187212 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8826be7e-3584-4f14-b24c-4cee4bc38a32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94bd01a2aa764b6583fd477015f56c82'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:36.187920 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8826be7e-3584-4f14-b24c-4cee4bc38a32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94bd01a2aa764b6583fd477015f56c82'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:36.201887 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8826be7e-3584-4f14-b24c-4cee4bc38a32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94bd01a2aa764b6583fd477015f56c82'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:36.202675 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8826be7e-3584-4f14-b24c-4cee4bc38a32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94bd01a2aa764b6583fd477015f56c82'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:36.219624 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1253/2527] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:45:36 2022] DELETE /identity/v3/users/94bd01a2aa764b6583fd477015f56c82 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:36.224726 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba0039-ba79-4648-9ad5-42f32f0c175f None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:36.225019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba0039-ba79-4648-9ad5-42f32f0c175f None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:36.225305 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba0039-ba79-4648-9ad5-42f32f0c175f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:36.248288 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-2eba0039-ba79-4648-9ad5-42f32f0c175f None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:36.249579 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-2eba0039-ba79-4648-9ad5-42f32f0c175f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:36.285937 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:36.319161 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1275/2528] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:36.329927 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826be7e-3584-4f14-b24c-4cee4bc38a32 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:36.331596 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8826be7e-3584-4f14-b24c-4cee4bc38a32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:36.392158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826be7e-3584-4f14-b24c-4cee4bc38a32 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:36.396260 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1b391-a772-446a-b089-8ff8d0aca745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:36.396908 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1b391-a772-446a-b089-8ff8d0aca745 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:36.397193 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1b391-a772-446a-b089-8ff8d0aca745 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:36.397471 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1b391-a772-446a-b089-8ff8d0aca745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:36.408185 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1b391-a772-446a-b089-8ff8d0aca745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:36.409652 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1b391-a772-446a-b089-8ff8d0aca745 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:36.419639 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1254/2529] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:45:36 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:45:36.720187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba0039-ba79-4648-9ad5-42f32f0c175f None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:36.721849 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-2eba0039-ba79-4648-9ad5-42f32f0c175f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:36.735318 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba0039-ba79-4648-9ad5-42f32f0c175f None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:36.740907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81639d2-7a5d-4a03-9aa9-e8de18863968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:36.741581 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81639d2-7a5d-4a03-9aa9-e8de18863968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:36.741864 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81639d2-7a5d-4a03-9aa9-e8de18863968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:36.742121 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81639d2-7a5d-4a03-9aa9-e8de18863968 admin admin] PATH_INFO: `/v3/projects/7b99cd04920d411984cf8efcc9458967` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:36.743327 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81639d2-7a5d-4a03-9aa9-e8de18863968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b99cd04920d411984cf8efcc9458967)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:36.745610 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81639d2-7a5d-4a03-9aa9-e8de18863968 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:36.796351 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-d81639d2-7a5d-4a03-9aa9-e8de18863968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b99cd04920d411984cf8efcc9458967', 'request_id': 'req-d81639d2-7a5d-4a03-9aa9-e8de18863968'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:36.805739 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-d81639d2-7a5d-4a03-9aa9-e8de18863968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b99cd04920d411984cf8efcc9458967', 'request_id': 'req-d81639d2-7a5d-4a03-9aa9-e8de18863968'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:36.813731 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-d81639d2-7a5d-4a03-9aa9-e8de18863968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b99cd04920d411984cf8efcc9458967', 'request_id': 'req-d81639d2-7a5d-4a03-9aa9-e8de18863968'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:36.832513 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-d81639d2-7a5d-4a03-9aa9-e8de18863968 admin admin] The token cache is being invalidate because project 7b99cd04920d411984cf8efcc9458967 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:45:36.832666 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-d81639d2-7a5d-4a03-9aa9-e8de18863968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:36.833916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-d81639d2-7a5d-4a03-9aa9-e8de18863968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:36.837917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1276/2530] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:45:36 2022] DELETE /identity/v3/projects/7b99cd04920d411984cf8efcc9458967 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:36.844198 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1b391-a772-446a-b089-8ff8d0aca745 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:36.857495 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1b391-a772-446a-b089-8ff8d0aca745 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:36.916069 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:36.925808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4728f-4f49-46e7-b46c-9bf05c868df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:36.926517 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4728f-4f49-46e7-b46c-9bf05c868df6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:36.926986 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4728f-4f49-46e7-b46c-9bf05c868df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:36.927529 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4728f-4f49-46e7-b46c-9bf05c868df6 admin admin] PATH_INFO: `/v3/users/5fa54b1103a64a25a0be0bf03c2f1dcc` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:36.928956 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4728f-4f49-46e7-b46c-9bf05c868df6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fa54b1103a64a25a0be0bf03c2f1dcc)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:36.930956 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4728f-4f49-46e7-b46c-9bf05c868df6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:36.981231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-eda4728f-4f49-46e7-b46c-9bf05c868df6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fa54b1103a64a25a0be0bf03c2f1dcc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:36.988262 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-eda4728f-4f49-46e7-b46c-9bf05c868df6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fa54b1103a64a25a0be0bf03c2f1dcc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:36.989062 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-eda4728f-4f49-46e7-b46c-9bf05c868df6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fa54b1103a64a25a0be0bf03c2f1dcc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:37.003302 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-eda4728f-4f49-46e7-b46c-9bf05c868df6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fa54b1103a64a25a0be0bf03c2f1dcc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:37.004129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-eda4728f-4f49-46e7-b46c-9bf05c868df6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fa54b1103a64a25a0be0bf03c2f1dcc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:37.019273 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1255/2531] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:45:36 2022] DELETE /identity/v3/users/5fa54b1103a64a25a0be0bf03c2f1dcc => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:37.325870 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81639d2-7a5d-4a03-9aa9-e8de18863968 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:37.327661 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d81639d2-7a5d-4a03-9aa9-e8de18863968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:37.349194 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81639d2-7a5d-4a03-9aa9-e8de18863968 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:37.420563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e64ad7-04b4-4cbb-ac04-4d86728512bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:37.421247 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e64ad7-04b4-4cbb-ac04-4d86728512bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:37.421529 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e64ad7-04b4-4cbb-ac04-4d86728512bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:37.421812 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e64ad7-04b4-4cbb-ac04-4d86728512bf admin admin] PATH_INFO: `/v3/projects/e7c310eb73944c33ba13d12dc6d33732` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:37.423045 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e64ad7-04b4-4cbb-ac04-4d86728512bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7c310eb73944c33ba13d12dc6d33732)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:37.425043 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e64ad7-04b4-4cbb-ac04-4d86728512bf admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:37.477645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-64e64ad7-04b4-4cbb-ac04-4d86728512bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7c310eb73944c33ba13d12dc6d33732', 'request_id': 'req-64e64ad7-04b4-4cbb-ac04-4d86728512bf'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:37.486316 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-64e64ad7-04b4-4cbb-ac04-4d86728512bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7c310eb73944c33ba13d12dc6d33732', 'request_id': 'req-64e64ad7-04b4-4cbb-ac04-4d86728512bf'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:37.490414 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:37.494061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-64e64ad7-04b4-4cbb-ac04-4d86728512bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7c310eb73944c33ba13d12dc6d33732', 'request_id': 'req-64e64ad7-04b4-4cbb-ac04-4d86728512bf'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:37.509612 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-64e64ad7-04b4-4cbb-ac04-4d86728512bf admin admin] The token cache is being invalidate because project e7c310eb73944c33ba13d12dc6d33732 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:45:37.510124 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-64e64ad7-04b4-4cbb-ac04-4d86728512bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:37.511098 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-64e64ad7-04b4-4cbb-ac04-4d86728512bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:37.512946 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1277/2532] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:45:37 2022] DELETE /identity/v3/projects/e7c310eb73944c33ba13d12dc6d33732 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:37.518434 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de3fe7-448c-4169-9b11-66b92be95133 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:37.518776 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de3fe7-448c-4169-9b11-66b92be95133 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:37.519092 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de3fe7-448c-4169-9b11-66b92be95133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:37.542940 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-95de3fe7-448c-4169-9b11-66b92be95133 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:37.544393 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-95de3fe7-448c-4169-9b11-66b92be95133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:37.631846 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1256/2533] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:37 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:37.633573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e9155-babf-42b1-998f-1143ab26f943 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:37.634054 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e9155-babf-42b1-998f-1143ab26f943 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:37.634444 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e9155-babf-42b1-998f-1143ab26f943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:37.660180 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-e97e9155-babf-42b1-998f-1143ab26f943 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:37.662714 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-e97e9155-babf-42b1-998f-1143ab26f943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:37.674407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1278/2534] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:37 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:37.679408 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de3fe7-448c-4169-9b11-66b92be95133 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:37.690792 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de3fe7-448c-4169-9b11-66b92be95133 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:37.697036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516ad0b-e51d-4366-a673-e14bbf907365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:37.697835 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516ad0b-e51d-4366-a673-e14bbf907365 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:37.698143 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516ad0b-e51d-4366-a673-e14bbf907365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:37.698484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516ad0b-e51d-4366-a673-e14bbf907365 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:37.699130 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f516ad0b-e51d-4366-a673-e14bbf907365 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:45:37.699439 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f516ad0b-e51d-4366-a673-e14bbf907365 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:37.701129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f516ad0b-e51d-4366-a673-e14bbf907365 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:37.721558 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1257/2535] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:45:37 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:45:37.726866 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e9155-babf-42b1-998f-1143ab26f943 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:37.747708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e9155-babf-42b1-998f-1143ab26f943 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:37.769529 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a6778-543f-4d37-900b-56039afb83c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:37.769529 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a6778-543f-4d37-900b-56039afb83c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:37.769529 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a6778-543f-4d37-900b-56039afb83c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:37.769529 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a6778-543f-4d37-900b-56039afb83c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:37.769920 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a6778-543f-4d37-900b-56039afb83c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:37.769920 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a6778-543f-4d37-900b-56039afb83c5 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:37.792771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1279/2536] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:45:37 2022] POST /identity/v3/projects => generated 366 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:37.797339 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516ad0b-e51d-4366-a673-e14bbf907365 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:37.808018 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516ad0b-e51d-4366-a673-e14bbf907365 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:37.868876 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df3845-ee5e-45af-b92f-03abc2f81349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:37.869840 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df3845-ee5e-45af-b92f-03abc2f81349 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:37.869840 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df3845-ee5e-45af-b92f-03abc2f81349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:37.870083 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df3845-ee5e-45af-b92f-03abc2f81349 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:37.870775 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df3845-ee5e-45af-b92f-03abc2f81349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:37.872862 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df3845-ee5e-45af-b92f-03abc2f81349 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:37.884327 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:37.917797 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1258/2537] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:45:37 2022] POST /identity/v3/users => generated 284 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:37.919229 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a6778-543f-4d37-900b-56039afb83c5 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:37.930877 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a6778-543f-4d37-900b-56039afb83c5 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:37.935699 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c147b5-6fa6-46de-bb6f-b29ff0164ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:37.936373 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c147b5-6fa6-46de-bb6f-b29ff0164ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:37.936655 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c147b5-6fa6-46de-bb6f-b29ff0164ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:37.936928 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c147b5-6fa6-46de-bb6f-b29ff0164ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:37.937670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c147b5-6fa6-46de-bb6f-b29ff0164ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:37.939298 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c147b5-6fa6-46de-bb6f-b29ff0164ba4 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:37.950115 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1280/2538] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:37 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:37.954718 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df3845-ee5e-45af-b92f-03abc2f81349 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:37.966469 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df3845-ee5e-45af-b92f-03abc2f81349 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:37.973615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6e773-5e76-474c-9c25-2d0af7496794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:37.974133 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6e773-5e76-474c-9c25-2d0af7496794 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:37.974455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6e773-5e76-474c-9c25-2d0af7496794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:37.974781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6e773-5e76-474c-9c25-2d0af7496794 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:37.975600 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa6e773-5e76-474c-9c25-2d0af7496794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:37.977532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa6e773-5e76-474c-9c25-2d0af7496794 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:37.989941 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1259/2539] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:37 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:37.994489 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c147b5-6fa6-46de-bb6f-b29ff0164ba4 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:38.005459 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c147b5-6fa6-46de-bb6f-b29ff0164ba4 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:38.010751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29894d6-1309-4b0b-b94a-0688cf99eda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:38.011442 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29894d6-1309-4b0b-b94a-0688cf99eda9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:38.011726 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29894d6-1309-4b0b-b94a-0688cf99eda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:38.012120 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29894d6-1309-4b0b-b94a-0688cf99eda9 admin admin] PATH_INFO: `/v3/projects/4201b11909c34f4484d952fe86e87287/users/90067d88c83140009457a8b478f55e58/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:38.027420 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29894d6-1309-4b0b-b94a-0688cf99eda9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4201b11909c34f4484d952fe86e87287, user_id=90067d88c83140009457a8b478f55e58, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:38.030156 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29894d6-1309-4b0b-b94a-0688cf99eda9 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:38.045694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1281/2540] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:37 2022] PUT /identity/v3/projects/4201b11909c34f4484d952fe86e87287/users/90067d88c83140009457a8b478f55e58/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:38.056612 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227af12-38f1-4254-9e17-1da48739176b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:38.056902 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227af12-38f1-4254-9e17-1da48739176b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:38.057260 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227af12-38f1-4254-9e17-1da48739176b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:38.086853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2227af12-38f1-4254-9e17-1da48739176b None None] MFA Rules not processed for user `90067d88c83140009457a8b478f55e58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:38.088155 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2227af12-38f1-4254-9e17-1da48739176b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:38.155637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1260/2541] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:38.160571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29894d6-1309-4b0b-b94a-0688cf99eda9 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:38.226753 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29894d6-1309-4b0b-b94a-0688cf99eda9 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:38.233197 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d408a-098c-4b59-8406-ec8fa8846828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:38.233905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d408a-098c-4b59-8406-ec8fa8846828 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:38.234188 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d408a-098c-4b59-8406-ec8fa8846828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:38.234456 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d408a-098c-4b59-8406-ec8fa8846828 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:38.235230 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d408a-098c-4b59-8406-ec8fa8846828 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:38.237333 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d408a-098c-4b59-8406-ec8fa8846828 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:38.269919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1282/2542] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:45:38 2022] POST /identity/v3/projects => generated 366 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:38.275055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227af12-38f1-4254-9e17-1da48739176b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:38.289218 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227af12-38f1-4254-9e17-1da48739176b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:38.359200 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f38bb7-3905-4de7-8032-a44434ebea31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:38.359882 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f38bb7-3905-4de7-8032-a44434ebea31 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:38.360228 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f38bb7-3905-4de7-8032-a44434ebea31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:38.360643 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f38bb7-3905-4de7-8032-a44434ebea31 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:38.361346 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f38bb7-3905-4de7-8032-a44434ebea31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:38.363453 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f38bb7-3905-4de7-8032-a44434ebea31 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:38.386374 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1261/2543] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:45:38 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:38.390965 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d408a-098c-4b59-8406-ec8fa8846828 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:38.402111 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d408a-098c-4b59-8406-ec8fa8846828 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:38.408520 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574aa757-a32b-42c8-a918-6105c8b3e14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:38.409106 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574aa757-a32b-42c8-a918-6105c8b3e14a admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:38.409390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574aa757-a32b-42c8-a918-6105c8b3e14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:38.409670 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574aa757-a32b-42c8-a918-6105c8b3e14a admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:38.410449 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574aa757-a32b-42c8-a918-6105c8b3e14a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:38.412185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574aa757-a32b-42c8-a918-6105c8b3e14a admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:38.423846 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1283/2544] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:38 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:38.428741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f38bb7-3905-4de7-8032-a44434ebea31 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:38.439231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f38bb7-3905-4de7-8032-a44434ebea31 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:38.444978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b18a93-a09d-41be-873a-45a88a93ac9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:38.445674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b18a93-a09d-41be-873a-45a88a93ac9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:38.445957 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b18a93-a09d-41be-873a-45a88a93ac9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:38.446235 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b18a93-a09d-41be-873a-45a88a93ac9d admin admin] PATH_INFO: `/v3/projects/f2a78892ad3948939cf0fd1afd68e107/users/9522e024f87846698cae72175201c01e/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:38.463987 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b18a93-a09d-41be-873a-45a88a93ac9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2a78892ad3948939cf0fd1afd68e107, user_id=9522e024f87846698cae72175201c01e, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:38.468421 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b18a93-a09d-41be-873a-45a88a93ac9d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:38.487601 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1262/2545] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:38 2022] PUT /identity/v3/projects/f2a78892ad3948939cf0fd1afd68e107/users/9522e024f87846698cae72175201c01e/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:38.492295 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574aa757-a32b-42c8-a918-6105c8b3e14a admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:38.502916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574aa757-a32b-42c8-a918-6105c8b3e14a admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:38.562277 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb8f77-8efa-448c-87d7-ac338ee23437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:38.562917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb8f77-8efa-448c-87d7-ac338ee23437 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:38.563202 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb8f77-8efa-448c-87d7-ac338ee23437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:38.563475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb8f77-8efa-448c-87d7-ac338ee23437 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:38.564225 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb8f77-8efa-448c-87d7-ac338ee23437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:38.565869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb8f77-8efa-448c-87d7-ac338ee23437 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:38.567515 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:38.579612 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1284/2546] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:38 2022] GET /identity/v3/roles => generated 1427 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:38.583000 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b18a93-a09d-41be-873a-45a88a93ac9d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:38.594809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b18a93-a09d-41be-873a-45a88a93ac9d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:38.599871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e125e4-64dc-4355-a5e4-dbe2e76ca1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:38.600595 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e125e4-64dc-4355-a5e4-dbe2e76ca1d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:38.600877 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e125e4-64dc-4355-a5e4-dbe2e76ca1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:38.601153 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e125e4-64dc-4355-a5e4-dbe2e76ca1d1 admin admin] PATH_INFO: `/v3/projects/f2a78892ad3948939cf0fd1afd68e107/users/9522e024f87846698cae72175201c01e/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:38.603255 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e125e4-64dc-4355-a5e4-dbe2e76ca1d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2a78892ad3948939cf0fd1afd68e107, user_id=9522e024f87846698cae72175201c01e, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:38.605994 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e125e4-64dc-4355-a5e4-dbe2e76ca1d1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:38.617614 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1263/2547] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:38 2022] PUT /identity/v3/projects/f2a78892ad3948939cf0fd1afd68e107/users/9522e024f87846698cae72175201c01e/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:38.630099 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466f84d-465c-48a4-8cef-a122e4852aff None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:38.630444 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466f84d-465c-48a4-8cef-a122e4852aff None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:38.630779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466f84d-465c-48a4-8cef-a122e4852aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:38.663075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-c466f84d-465c-48a4-8cef-a122e4852aff None None] MFA Rules not processed for user `9522e024f87846698cae72175201c01e`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:38.664278 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c466f84d-465c-48a4-8cef-a122e4852aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:38.753971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1285/2548] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:38.766309 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e125e4-64dc-4355-a5e4-dbe2e76ca1d1 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:38.768376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-77e125e4-64dc-4355-a5e4-dbe2e76ca1d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:38.801197 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:38.846015 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e125e4-64dc-4355-a5e4-dbe2e76ca1d1 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:38.850156 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e8233-9948-4149-b045-2f8725e90e99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:38.850731 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e8233-9948-4149-b045-2f8725e90e99 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:38.851009 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e8233-9948-4149-b045-2f8725e90e99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:38.851285 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e8233-9948-4149-b045-2f8725e90e99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:38.861862 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e8233-9948-4149-b045-2f8725e90e99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:38.863374 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e8233-9948-4149-b045-2f8725e90e99 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:38.873851 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1264/2549] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:45:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:45:39.348931 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466f84d-465c-48a4-8cef-a122e4852aff None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:39.423364 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466f84d-465c-48a4-8cef-a122e4852aff None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:39.428196 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ba27f-4922-4ab2-ad2f-5a383e019e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:39.428785 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ba27f-4922-4ab2-ad2f-5a383e019e7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:39.429072 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ba27f-4922-4ab2-ad2f-5a383e019e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:39.429353 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ba27f-4922-4ab2-ad2f-5a383e019e7a admin admin] PATH_INFO: `/v3/users/90067d88c83140009457a8b478f55e58` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:39.430575 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ba27f-4922-4ab2-ad2f-5a383e019e7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=90067d88c83140009457a8b478f55e58)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:39.432661 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ba27f-4922-4ab2-ad2f-5a383e019e7a admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:39.484813 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-2d9ba27f-4922-4ab2-ad2f-5a383e019e7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90067d88c83140009457a8b478f55e58'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:39.485510 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-2d9ba27f-4922-4ab2-ad2f-5a383e019e7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90067d88c83140009457a8b478f55e58'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:39.497302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-2d9ba27f-4922-4ab2-ad2f-5a383e019e7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90067d88c83140009457a8b478f55e58'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:39.504197 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-2d9ba27f-4922-4ab2-ad2f-5a383e019e7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90067d88c83140009457a8b478f55e58'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:39.513467 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-2d9ba27f-4922-4ab2-ad2f-5a383e019e7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90067d88c83140009457a8b478f55e58'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:39.515873 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:39.521501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1286/2550] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:45:39 2022] DELETE /identity/v3/users/90067d88c83140009457a8b478f55e58 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:39.527332 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0cdc4-940f-45bc-8ff4-da97a5478377 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:39.527797 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0cdc4-940f-45bc-8ff4-da97a5478377 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:39.528223 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0cdc4-940f-45bc-8ff4-da97a5478377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:39.550809 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-aac0cdc4-940f-45bc-8ff4-da97a5478377 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:39.552127 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-aac0cdc4-940f-45bc-8ff4-da97a5478377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:39.616890 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1265/2551] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:39 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:39.627534 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ba27f-4922-4ab2-ad2f-5a383e019e7a admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:39.629265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-2d9ba27f-4922-4ab2-ad2f-5a383e019e7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:39.704799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ba27f-4922-4ab2-ad2f-5a383e019e7a admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:39.708795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d26c1f-1f3d-4c45-8b9f-685a131a1196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:39.709445 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d26c1f-1f3d-4c45-8b9f-685a131a1196 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:39.709727 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d26c1f-1f3d-4c45-8b9f-685a131a1196 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:39.710008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d26c1f-1f3d-4c45-8b9f-685a131a1196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:39.720583 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d26c1f-1f3d-4c45-8b9f-685a131a1196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:39.722049 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d26c1f-1f3d-4c45-8b9f-685a131a1196 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:39.732058 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1287/2552] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:45:39 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:45:40.029740 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0cdc4-940f-45bc-8ff4-da97a5478377 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:40.031411 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-aac0cdc4-940f-45bc-8ff4-da97a5478377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:40.045371 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0cdc4-940f-45bc-8ff4-da97a5478377 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:40.051920 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270df8ea-92ea-490f-9b36-38e440232d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:40.051920 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270df8ea-92ea-490f-9b36-38e440232d5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:40.052272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270df8ea-92ea-490f-9b36-38e440232d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:40.052272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270df8ea-92ea-490f-9b36-38e440232d5f admin admin] PATH_INFO: `/v3/projects/4201b11909c34f4484d952fe86e87287` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:40.053614 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270df8ea-92ea-490f-9b36-38e440232d5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4201b11909c34f4484d952fe86e87287)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:40.055604 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270df8ea-92ea-490f-9b36-38e440232d5f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:40.113760 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-270df8ea-92ea-490f-9b36-38e440232d5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4201b11909c34f4484d952fe86e87287', 'request_id': 'req-270df8ea-92ea-490f-9b36-38e440232d5f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:40.120599 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-270df8ea-92ea-490f-9b36-38e440232d5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4201b11909c34f4484d952fe86e87287', 'request_id': 'req-270df8ea-92ea-490f-9b36-38e440232d5f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:40.126632 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-270df8ea-92ea-490f-9b36-38e440232d5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4201b11909c34f4484d952fe86e87287', 'request_id': 'req-270df8ea-92ea-490f-9b36-38e440232d5f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:40.140702 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-270df8ea-92ea-490f-9b36-38e440232d5f admin admin] The token cache is being invalidate because project 4201b11909c34f4484d952fe86e87287 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:45:40.140994 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-270df8ea-92ea-490f-9b36-38e440232d5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:40.141728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-270df8ea-92ea-490f-9b36-38e440232d5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:40.143423 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1266/2553] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:45:40 2022] DELETE /identity/v3/projects/4201b11909c34f4484d952fe86e87287 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:40.147866 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d26c1f-1f3d-4c45-8b9f-685a131a1196 service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:40.159238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d26c1f-1f3d-4c45-8b9f-685a131a1196 service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:40.219896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558d76f-11b5-43b7-829e-e6ac3b93af23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:40.220735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558d76f-11b5-43b7-829e-e6ac3b93af23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:40.221086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558d76f-11b5-43b7-829e-e6ac3b93af23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:40.221300 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558d76f-11b5-43b7-829e-e6ac3b93af23 admin admin] PATH_INFO: `/v3/users/9522e024f87846698cae72175201c01e` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:40.222524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a558d76f-11b5-43b7-829e-e6ac3b93af23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9522e024f87846698cae72175201c01e)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:40.224577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a558d76f-11b5-43b7-829e-e6ac3b93af23 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:40.275406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a558d76f-11b5-43b7-829e-e6ac3b93af23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9522e024f87846698cae72175201c01e'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:40.276424 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a558d76f-11b5-43b7-829e-e6ac3b93af23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9522e024f87846698cae72175201c01e'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:40.290277 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a558d76f-11b5-43b7-829e-e6ac3b93af23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9522e024f87846698cae72175201c01e'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:40.297684 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a558d76f-11b5-43b7-829e-e6ac3b93af23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9522e024f87846698cae72175201c01e'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:40.306134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-a558d76f-11b5-43b7-829e-e6ac3b93af23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9522e024f87846698cae72175201c01e'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:40.312677 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1288/2554] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:45:40 2022] DELETE /identity/v3/users/9522e024f87846698cae72175201c01e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:40.603713 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270df8ea-92ea-490f-9b36-38e440232d5f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:40.605442 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-270df8ea-92ea-490f-9b36-38e440232d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:40.619703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270df8ea-92ea-490f-9b36-38e440232d5f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:40.669149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:40.680840 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed0d0c-be5c-4f00-8565-d5a6a92177ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:40.681512 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed0d0c-be5c-4f00-8565-d5a6a92177ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:40.681805 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed0d0c-be5c-4f00-8565-d5a6a92177ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:40.682091 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed0d0c-be5c-4f00-8565-d5a6a92177ac admin admin] PATH_INFO: `/v3/projects/f2a78892ad3948939cf0fd1afd68e107` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:40.683485 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed0d0c-be5c-4f00-8565-d5a6a92177ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2a78892ad3948939cf0fd1afd68e107)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:40.685415 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed0d0c-be5c-4f00-8565-d5a6a92177ac admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:40.736267 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-30ed0d0c-be5c-4f00-8565-d5a6a92177ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2a78892ad3948939cf0fd1afd68e107', 'request_id': 'req-30ed0d0c-be5c-4f00-8565-d5a6a92177ac'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:40.742528 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-30ed0d0c-be5c-4f00-8565-d5a6a92177ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2a78892ad3948939cf0fd1afd68e107', 'request_id': 'req-30ed0d0c-be5c-4f00-8565-d5a6a92177ac'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:40.748343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-30ed0d0c-be5c-4f00-8565-d5a6a92177ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2a78892ad3948939cf0fd1afd68e107', 'request_id': 'req-30ed0d0c-be5c-4f00-8565-d5a6a92177ac'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:40.762660 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-30ed0d0c-be5c-4f00-8565-d5a6a92177ac admin admin] The token cache is being invalidate because project f2a78892ad3948939cf0fd1afd68e107 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:45:40.762949 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-30ed0d0c-be5c-4f00-8565-d5a6a92177ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:40.763595 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-30ed0d0c-be5c-4f00-8565-d5a6a92177ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:40.765214 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1267/2555] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:45:40 2022] DELETE /identity/v3/projects/f2a78892ad3948939cf0fd1afd68e107 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:40.770648 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3bbcd-4a63-45a4-bab8-e0be0997dda8 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:40.770962 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3bbcd-4a63-45a4-bab8-e0be0997dda8 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:40.771264 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3bbcd-4a63-45a4-bab8-e0be0997dda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:40.794340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-fbc3bbcd-4a63-45a4-bab8-e0be0997dda8 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:40.795625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-fbc3bbcd-4a63-45a4-bab8-e0be0997dda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:40.814736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:40.862306 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1289/2556] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:40 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:40.870892 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a269d7-2331-43ab-b678-fd9c6ff47246 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:40.871154 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a269d7-2331-43ab-b678-fd9c6ff47246 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:40.871499 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a269d7-2331-43ab-b678-fd9c6ff47246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:40.893381 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-67a269d7-2331-43ab-b678-fd9c6ff47246 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:40.894655 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-67a269d7-2331-43ab-b678-fd9c6ff47246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:40.903110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1268/2557] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:40 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:40.907709 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3bbcd-4a63-45a4-bab8-e0be0997dda8 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:40.919349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3bbcd-4a63-45a4-bab8-e0be0997dda8 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:40.924804 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075cae8-ddab-4dee-ae0a-66fc8ce21752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:40.925460 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075cae8-ddab-4dee-ae0a-66fc8ce21752 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:40.925750 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075cae8-ddab-4dee-ae0a-66fc8ce21752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:40.926039 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075cae8-ddab-4dee-ae0a-66fc8ce21752 admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:40.926762 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6075cae8-ddab-4dee-ae0a-66fc8ce21752 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:45:40.927078 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6075cae8-ddab-4dee-ae0a-66fc8ce21752 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:40.928782 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6075cae8-ddab-4dee-ae0a-66fc8ce21752 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:40.942649 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1290/2558] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:45:40 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:45:40.947856 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a269d7-2331-43ab-b678-fd9c6ff47246 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:40.963245 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a269d7-2331-43ab-b678-fd9c6ff47246 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:40.969626 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb553031-b296-4a87-9944-c171071722ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:40.970278 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb553031-b296-4a87-9944-c171071722ad admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:40.970561 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb553031-b296-4a87-9944-c171071722ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:40.970853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb553031-b296-4a87-9944-c171071722ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:40.971562 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb553031-b296-4a87-9944-c171071722ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:40.973567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb553031-b296-4a87-9944-c171071722ad admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:41.011896 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1269/2559] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:45:40 2022] POST /identity/v3/projects => generated 388 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:41.017936 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075cae8-ddab-4dee-ae0a-66fc8ce21752 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:41.029111 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075cae8-ddab-4dee-ae0a-66fc8ce21752 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:41.088935 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afffe38-e2ff-4be2-90e7-67e179011fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:41.089578 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afffe38-e2ff-4be2-90e7-67e179011fc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:41.089858 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afffe38-e2ff-4be2-90e7-67e179011fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:41.090128 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afffe38-e2ff-4be2-90e7-67e179011fc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:41.090803 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afffe38-e2ff-4be2-90e7-67e179011fc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:41.092955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afffe38-e2ff-4be2-90e7-67e179011fc6 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:41.118119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1291/2560] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:45:41 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:45:41.122750 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb553031-b296-4a87-9944-c171071722ad admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:41.134420 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb553031-b296-4a87-9944-c171071722ad admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:41.140465 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586c507-8c70-4008-bdad-1545fdd12998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:41.141128 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586c507-8c70-4008-bdad-1545fdd12998 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:41.141413 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586c507-8c70-4008-bdad-1545fdd12998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:41.141688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586c507-8c70-4008-bdad-1545fdd12998 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:41.142409 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b586c507-8c70-4008-bdad-1545fdd12998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:41.144122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b586c507-8c70-4008-bdad-1545fdd12998 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:41.155570 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1270/2561] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:41 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:41.160323 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afffe38-e2ff-4be2-90e7-67e179011fc6 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:41.171466 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afffe38-e2ff-4be2-90e7-67e179011fc6 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:41.177278 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25660bf-ebe2-4082-b3b6-fd97c9bfa426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:41.178034 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25660bf-ebe2-4082-b3b6-fd97c9bfa426 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:41.178322 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25660bf-ebe2-4082-b3b6-fd97c9bfa426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:41.178580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25660bf-ebe2-4082-b3b6-fd97c9bfa426 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:41.179291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25660bf-ebe2-4082-b3b6-fd97c9bfa426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:41.181264 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25660bf-ebe2-4082-b3b6-fd97c9bfa426 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:41.193695 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1292/2562] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:41 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:45:41.198341 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586c507-8c70-4008-bdad-1545fdd12998 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:41.209448 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586c507-8c70-4008-bdad-1545fdd12998 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:41.215201 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70e97d-d79c-4552-a706-1286498fa0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:41.215916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70e97d-d79c-4552-a706-1286498fa0bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:41.216271 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70e97d-d79c-4552-a706-1286498fa0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:41.216524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70e97d-d79c-4552-a706-1286498fa0bf admin admin] PATH_INFO: `/v3/projects/83ee858d2cb8468d97369d353f0e250e/users/9952ba463ef54928be1fe94de6cb0ee0/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:41.232480 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb70e97d-d79c-4552-a706-1286498fa0bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=83ee858d2cb8468d97369d353f0e250e, user_id=9952ba463ef54928be1fe94de6cb0ee0, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:41.235188 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb70e97d-d79c-4552-a706-1286498fa0bf admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:41.249686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1271/2563] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:41 2022] PUT /identity/v3/projects/83ee858d2cb8468d97369d353f0e250e/users/9952ba463ef54928be1fe94de6cb0ee0/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:41.262362 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea4f80-3642-47f9-b564-6e20d0548972 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:41.262577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea4f80-3642-47f9-b564-6e20d0548972 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:41.262856 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea4f80-3642-47f9-b564-6e20d0548972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:41.295595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-aaea4f80-3642-47f9-b564-6e20d0548972 None None] MFA Rules not processed for user `9952ba463ef54928be1fe94de6cb0ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:41.296876 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-aaea4f80-3642-47f9-b564-6e20d0548972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:41.366652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1293/2564] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:41 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:41.371689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70e97d-d79c-4552-a706-1286498fa0bf admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:41.437673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70e97d-d79c-4552-a706-1286498fa0bf admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:41.443435 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f5cbc-7e1e-48af-88ea-26830f1430f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:41.444177 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f5cbc-7e1e-48af-88ea-26830f1430f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:41.444464 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f5cbc-7e1e-48af-88ea-26830f1430f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:41.444740 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f5cbc-7e1e-48af-88ea-26830f1430f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:41.445425 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f5cbc-7e1e-48af-88ea-26830f1430f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:41.447379 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f5cbc-7e1e-48af-88ea-26830f1430f3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:41.479714 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1272/2565] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:45:41 2022] POST /identity/v3/projects => generated 386 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:41.484419 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea4f80-3642-47f9-b564-6e20d0548972 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:41.499358 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea4f80-3642-47f9-b564-6e20d0548972 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:41.514524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:41.563623 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b9083-1086-40dd-aeab-975dabeaf2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:41.564308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b9083-1086-40dd-aeab-975dabeaf2ff admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:41.564592 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b9083-1086-40dd-aeab-975dabeaf2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:41.564875 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b9083-1086-40dd-aeab-975dabeaf2ff admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:41.565575 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6b9083-1086-40dd-aeab-975dabeaf2ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:41.567585 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6b9083-1086-40dd-aeab-975dabeaf2ff admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:41.593005 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1294/2566] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:45:41 2022] POST /identity/v3/users => generated 300 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:41.598100 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f5cbc-7e1e-48af-88ea-26830f1430f3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:41.604040 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:41.610923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f5cbc-7e1e-48af-88ea-26830f1430f3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:41.617297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900ba9d-5693-4bd7-bf14-760405602c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:41.617938 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900ba9d-5693-4bd7-bf14-760405602c8c admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:41.618228 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900ba9d-5693-4bd7-bf14-760405602c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:41.618507 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900ba9d-5693-4bd7-bf14-760405602c8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:41.619232 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a900ba9d-5693-4bd7-bf14-760405602c8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:41.621009 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a900ba9d-5693-4bd7-bf14-760405602c8c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:41.633267 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1273/2567] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:41 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:41.638242 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b9083-1086-40dd-aeab-975dabeaf2ff admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:41.649070 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b9083-1086-40dd-aeab-975dabeaf2ff admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:41.654194 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432944e-9466-4309-b05d-8388192f3657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:41.654884 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432944e-9466-4309-b05d-8388192f3657 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:41.655165 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432944e-9466-4309-b05d-8388192f3657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:41.655465 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432944e-9466-4309-b05d-8388192f3657 admin admin] PATH_INFO: `/v3/projects/363b83cee7c84087a159979f9516c542/users/3f52943959ed4052b4d1b75d22a65311/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:41.678588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6432944e-9466-4309-b05d-8388192f3657 admin admin] RBAC: Authorizing `identity:create_grant(project_id=363b83cee7c84087a159979f9516c542, user_id=3f52943959ed4052b4d1b75d22a65311, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:41.681389 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6432944e-9466-4309-b05d-8388192f3657 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:41.697124 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1295/2568] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:41 2022] PUT /identity/v3/projects/363b83cee7c84087a159979f9516c542/users/3f52943959ed4052b4d1b75d22a65311/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:41.701562 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900ba9d-5693-4bd7-bf14-760405602c8c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:41.712634 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900ba9d-5693-4bd7-bf14-760405602c8c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:41.775731 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418910e8-a831-429e-83f8-9c55df92f3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:41.776330 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418910e8-a831-429e-83f8-9c55df92f3f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:41.776616 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418910e8-a831-429e-83f8-9c55df92f3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:41.776894 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418910e8-a831-429e-83f8-9c55df92f3f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:41.777611 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418910e8-a831-429e-83f8-9c55df92f3f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:41.779283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418910e8-a831-429e-83f8-9c55df92f3f1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:41.791302 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1274/2569] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:41 2022] GET /identity/v3/roles => generated 1427 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:41.796531 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432944e-9466-4309-b05d-8388192f3657 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:41.808895 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432944e-9466-4309-b05d-8388192f3657 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:41.813844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee78de63-08a1-4f5a-810c-cd04a5db1551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:41.814537 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78de63-08a1-4f5a-810c-cd04a5db1551 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:41.814817 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78de63-08a1-4f5a-810c-cd04a5db1551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:41.815096 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78de63-08a1-4f5a-810c-cd04a5db1551 admin admin] PATH_INFO: `/v3/projects/363b83cee7c84087a159979f9516c542/users/3f52943959ed4052b4d1b75d22a65311/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:41.817227 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee78de63-08a1-4f5a-810c-cd04a5db1551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=363b83cee7c84087a159979f9516c542, user_id=3f52943959ed4052b4d1b75d22a65311, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:41.819905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee78de63-08a1-4f5a-810c-cd04a5db1551 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:41.831335 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1296/2570] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:41 2022] PUT /identity/v3/projects/363b83cee7c84087a159979f9516c542/users/3f52943959ed4052b4d1b75d22a65311/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:41.842462 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da80ff9-4096-4528-bf12-428ef9ff6ce5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:41.842778 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da80ff9-4096-4528-bf12-428ef9ff6ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:41.843039 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da80ff9-4096-4528-bf12-428ef9ff6ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:41.874241 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-3da80ff9-4096-4528-bf12-428ef9ff6ce5 None None] MFA Rules not processed for user `3f52943959ed4052b4d1b75d22a65311`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:41.875524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3da80ff9-4096-4528-bf12-428ef9ff6ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:41.951990 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1275/2571] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:41 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:41.964418 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee78de63-08a1-4f5a-810c-cd04a5db1551 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:41.966365 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-ee78de63-08a1-4f5a-810c-cd04a5db1551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:42.029514 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee78de63-08a1-4f5a-810c-cd04a5db1551 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:42.033250 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8709535-11e9-4c24-a8a1-fee8c216106d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:42.033905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8709535-11e9-4c24-a8a1-fee8c216106d service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:42.034189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8709535-11e9-4c24-a8a1-fee8c216106d service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:42.034470 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8709535-11e9-4c24-a8a1-fee8c216106d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:42.045051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8709535-11e9-4c24-a8a1-fee8c216106d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:42.046526 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8709535-11e9-4c24-a8a1-fee8c216106d service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:42.056520 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1297/2572] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:45:41 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:45:42.086572 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da80ff9-4096-4528-bf12-428ef9ff6ce5 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:42.154234 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da80ff9-4096-4528-bf12-428ef9ff6ce5 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:42.160264 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b5a27-c193-41a2-85ac-0ea3551966e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:42.160958 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b5a27-c193-41a2-85ac-0ea3551966e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:42.161258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b5a27-c193-41a2-85ac-0ea3551966e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:42.161520 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b5a27-c193-41a2-85ac-0ea3551966e0 admin admin] PATH_INFO: `/v3/users/9952ba463ef54928be1fe94de6cb0ee0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:42.162885 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b5a27-c193-41a2-85ac-0ea3551966e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9952ba463ef54928be1fe94de6cb0ee0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:42.164922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b5a27-c193-41a2-85ac-0ea3551966e0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:42.180448 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:42.221467 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1c2b5a27-c193-41a2-85ac-0ea3551966e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9952ba463ef54928be1fe94de6cb0ee0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:42.228449 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1c2b5a27-c193-41a2-85ac-0ea3551966e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9952ba463ef54928be1fe94de6cb0ee0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:42.229244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1c2b5a27-c193-41a2-85ac-0ea3551966e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9952ba463ef54928be1fe94de6cb0ee0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:42.242230 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1c2b5a27-c193-41a2-85ac-0ea3551966e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9952ba463ef54928be1fe94de6cb0ee0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:42.242925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1c2b5a27-c193-41a2-85ac-0ea3551966e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9952ba463ef54928be1fe94de6cb0ee0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:42.257638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1276/2573] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:45:42 2022] DELETE /identity/v3/users/9952ba463ef54928be1fe94de6cb0ee0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:45:42.263093 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f492f46-25a6-4631-ad99-1bbb0bb0c41c None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:42.263651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f492f46-25a6-4631-ad99-1bbb0bb0c41c None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:42.263651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f492f46-25a6-4631-ad99-1bbb0bb0c41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:42.284220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-6f492f46-25a6-4631-ad99-1bbb0bb0c41c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:42.285584 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6f492f46-25a6-4631-ad99-1bbb0bb0c41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:42.347458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1298/2574] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:42.358390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b5a27-c193-41a2-85ac-0ea3551966e0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:42.360139 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1c2b5a27-c193-41a2-85ac-0ea3551966e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:42.423008 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b5a27-c193-41a2-85ac-0ea3551966e0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:42.427547 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c2763-eb77-468e-8371-a8055f41fdbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:42.428210 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c2763-eb77-468e-8371-a8055f41fdbe service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:42.428496 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c2763-eb77-468e-8371-a8055f41fdbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:42.428777 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c2763-eb77-468e-8371-a8055f41fdbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:42.438980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c2763-eb77-468e-8371-a8055f41fdbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:42.440474 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c2763-eb77-468e-8371-a8055f41fdbe service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:42.450326 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1277/2575] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:45:42 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:45:42.744117 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f492f46-25a6-4631-ad99-1bbb0bb0c41c None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:42.745704 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6f492f46-25a6-4631-ad99-1bbb0bb0c41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:42.758794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f492f46-25a6-4631-ad99-1bbb0bb0c41c None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:42.764821 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e829c66-3f64-44d9-80cf-b9b13c442c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:42.766137 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e829c66-3f64-44d9-80cf-b9b13c442c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:42.766631 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e829c66-3f64-44d9-80cf-b9b13c442c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:42.766869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e829c66-3f64-44d9-80cf-b9b13c442c78 admin admin] PATH_INFO: `/v3/projects/83ee858d2cb8468d97369d353f0e250e` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:42.768162 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e829c66-3f64-44d9-80cf-b9b13c442c78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83ee858d2cb8468d97369d353f0e250e)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:42.770404 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e829c66-3f64-44d9-80cf-b9b13c442c78 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:42.779034 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:42.821185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-8e829c66-3f64-44d9-80cf-b9b13c442c78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83ee858d2cb8468d97369d353f0e250e', 'request_id': 'req-8e829c66-3f64-44d9-80cf-b9b13c442c78'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:42.831086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-8e829c66-3f64-44d9-80cf-b9b13c442c78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83ee858d2cb8468d97369d353f0e250e', 'request_id': 'req-8e829c66-3f64-44d9-80cf-b9b13c442c78'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:42.837403 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-8e829c66-3f64-44d9-80cf-b9b13c442c78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83ee858d2cb8468d97369d353f0e250e', 'request_id': 'req-8e829c66-3f64-44d9-80cf-b9b13c442c78'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:42.849245 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-8e829c66-3f64-44d9-80cf-b9b13c442c78 admin admin] The token cache is being invalidate because project 83ee858d2cb8468d97369d353f0e250e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:45:42.849534 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-8e829c66-3f64-44d9-80cf-b9b13c442c78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:42.850344 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-8e829c66-3f64-44d9-80cf-b9b13c442c78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:42.852063 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1299/2576] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:45:42 2022] DELETE /identity/v3/projects/83ee858d2cb8468d97369d353f0e250e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:42.856418 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c2763-eb77-468e-8371-a8055f41fdbe service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:42.867171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c2763-eb77-468e-8371-a8055f41fdbe service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:42.930014 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9329c7-60b3-4aac-9b94-e2b52742b0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:42.930572 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9329c7-60b3-4aac-9b94-e2b52742b0d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:42.930841 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9329c7-60b3-4aac-9b94-e2b52742b0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:42.931097 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9329c7-60b3-4aac-9b94-e2b52742b0d5 admin admin] PATH_INFO: `/v3/users/3f52943959ed4052b4d1b75d22a65311` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:42.932406 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9329c7-60b3-4aac-9b94-e2b52742b0d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f52943959ed4052b4d1b75d22a65311)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:42.934411 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9329c7-60b3-4aac-9b94-e2b52742b0d5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:42.978516 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:42.989542 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fd9329c7-60b3-4aac-9b94-e2b52742b0d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f52943959ed4052b4d1b75d22a65311'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:42.997026 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fd9329c7-60b3-4aac-9b94-e2b52742b0d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f52943959ed4052b4d1b75d22a65311'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:42.997724 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fd9329c7-60b3-4aac-9b94-e2b52742b0d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f52943959ed4052b4d1b75d22a65311'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:43.010678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fd9329c7-60b3-4aac-9b94-e2b52742b0d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f52943959ed4052b4d1b75d22a65311'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:43.011705 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-fd9329c7-60b3-4aac-9b94-e2b52742b0d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f52943959ed4052b4d1b75d22a65311'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:43.026979 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1278/2577] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:45:42 2022] DELETE /identity/v3/users/3f52943959ed4052b4d1b75d22a65311 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:43.315885 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e829c66-3f64-44d9-80cf-b9b13c442c78 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:43.317435 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-8e829c66-3f64-44d9-80cf-b9b13c442c78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:43.333818 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e829c66-3f64-44d9-80cf-b9b13c442c78 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:43.395052 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e9aac-8544-4711-826d-694be2818983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:43.395620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e9aac-8544-4711-826d-694be2818983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:43.395908 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e9aac-8544-4711-826d-694be2818983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:43.396230 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e9aac-8544-4711-826d-694be2818983 admin admin] PATH_INFO: `/v3/projects/363b83cee7c84087a159979f9516c542` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:43.397505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e9aac-8544-4711-826d-694be2818983 admin admin] RBAC: Authorizing `identity:delete_project(project_id=363b83cee7c84087a159979f9516c542)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:43.399608 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e9aac-8544-4711-826d-694be2818983 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:43.449134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-bc5e9aac-8544-4711-826d-694be2818983 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '363b83cee7c84087a159979f9516c542', 'request_id': 'req-bc5e9aac-8544-4711-826d-694be2818983'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:43.457956 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-bc5e9aac-8544-4711-826d-694be2818983 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '363b83cee7c84087a159979f9516c542', 'request_id': 'req-bc5e9aac-8544-4711-826d-694be2818983'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:43.464323 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-bc5e9aac-8544-4711-826d-694be2818983 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '363b83cee7c84087a159979f9516c542', 'request_id': 'req-bc5e9aac-8544-4711-826d-694be2818983'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:43.477153 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-bc5e9aac-8544-4711-826d-694be2818983 admin admin] The token cache is being invalidate because project 363b83cee7c84087a159979f9516c542 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:45:43.477608 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-bc5e9aac-8544-4711-826d-694be2818983 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:43.478126 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-bc5e9aac-8544-4711-826d-694be2818983 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:43.479859 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1300/2578] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:45:43 2022] DELETE /identity/v3/projects/363b83cee7c84087a159979f9516c542 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:43.486367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598c658-e2f8-4f5f-b239-e7124ef73e01 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:43.486367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598c658-e2f8-4f5f-b239-e7124ef73e01 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:43.486367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598c658-e2f8-4f5f-b239-e7124ef73e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:43.506495 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2598c658-e2f8-4f5f-b239-e7124ef73e01 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:43.507772 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2598c658-e2f8-4f5f-b239-e7124ef73e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:43.571455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1279/2579] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:43.579993 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfca684-e94d-4828-95e6-5826d67ff046 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:43.580301 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfca684-e94d-4828-95e6-5826d67ff046 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:43.580579 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfca684-e94d-4828-95e6-5826d67ff046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:43.601647 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-4dfca684-e94d-4828-95e6-5826d67ff046 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:43.603689 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-4dfca684-e94d-4828-95e6-5826d67ff046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:43.612460 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1301/2580] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:43.617240 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598c658-e2f8-4f5f-b239-e7124ef73e01 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:43.627829 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598c658-e2f8-4f5f-b239-e7124ef73e01 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:43.632972 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23d6ec-18de-47a6-991d-fd2401231fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:43.633600 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23d6ec-18de-47a6-991d-fd2401231fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:43.633881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23d6ec-18de-47a6-991d-fd2401231fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:43.634154 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23d6ec-18de-47a6-991d-fd2401231fa5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:43.634863 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23d6ec-18de-47a6-991d-fd2401231fa5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:45:43.635281 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23d6ec-18de-47a6-991d-fd2401231fa5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:43.636964 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23d6ec-18de-47a6-991d-fd2401231fa5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:43.649051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1280/2581] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:45:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:45:43.653780 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfca684-e94d-4828-95e6-5826d67ff046 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:43.663916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfca684-e94d-4828-95e6-5826d67ff046 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:43.669353 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80e96e-6798-47d7-b4db-fd7388b62335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:43.669993 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80e96e-6798-47d7-b4db-fd7388b62335 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:43.670278 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80e96e-6798-47d7-b4db-fd7388b62335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:43.670555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80e96e-6798-47d7-b4db-fd7388b62335 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:43.671327 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80e96e-6798-47d7-b4db-fd7388b62335 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:43.673342 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80e96e-6798-47d7-b4db-fd7388b62335 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:43.699406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:43.713027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1302/2582] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:45:43 2022] POST /identity/v3/projects => generated 380 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:43.714573 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23d6ec-18de-47a6-991d-fd2401231fa5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:43.725795 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23d6ec-18de-47a6-991d-fd2401231fa5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:43.789041 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae84fc-bb67-41db-966f-74baa702eaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:43.789723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae84fc-bb67-41db-966f-74baa702eaf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:43.790051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae84fc-bb67-41db-966f-74baa702eaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:43.790294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae84fc-bb67-41db-966f-74baa702eaf5 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:43.790978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae84fc-bb67-41db-966f-74baa702eaf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:43.793509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae84fc-bb67-41db-966f-74baa702eaf5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:43.817472 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1281/2583] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:45:43 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:43.821898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80e96e-6798-47d7-b4db-fd7388b62335 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:43.835517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80e96e-6798-47d7-b4db-fd7388b62335 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:43.840781 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c72988-a084-4194-b196-bc330e3ac349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:43.841439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c72988-a084-4194-b196-bc330e3ac349 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:43.841728 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c72988-a084-4194-b196-bc330e3ac349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:43.842008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c72988-a084-4194-b196-bc330e3ac349 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:43.842721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c72988-a084-4194-b196-bc330e3ac349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:43.844780 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c72988-a084-4194-b196-bc330e3ac349 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:43.856122 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1303/2584] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:43 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:45:43.860575 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae84fc-bb67-41db-966f-74baa702eaf5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:43.871549 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae84fc-bb67-41db-966f-74baa702eaf5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:43.877683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a8e87d-b9df-45e3-8308-bbeca8fc8cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:43.878331 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8e87d-b9df-45e3-8308-bbeca8fc8cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:43.878620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8e87d-b9df-45e3-8308-bbeca8fc8cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:43.878895 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8e87d-b9df-45e3-8308-bbeca8fc8cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:43.879616 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8e87d-b9df-45e3-8308-bbeca8fc8cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:43.881287 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8e87d-b9df-45e3-8308-bbeca8fc8cc7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:43.893180 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1282/2585] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:43 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:43.898080 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c72988-a084-4194-b196-bc330e3ac349 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:43.914360 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c72988-a084-4194-b196-bc330e3ac349 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:43.921390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d138dc9-71ed-4f41-ac9b-b2f228598538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:43.922250 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d138dc9-71ed-4f41-ac9b-b2f228598538 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:43.922680 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d138dc9-71ed-4f41-ac9b-b2f228598538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:43.923118 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d138dc9-71ed-4f41-ac9b-b2f228598538 admin admin] PATH_INFO: `/v3/projects/2274de7f43ec460cbab7b5062e7fc207/users/999da485e32b40e0acb68a44aa1f6821/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:43.943044 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d138dc9-71ed-4f41-ac9b-b2f228598538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2274de7f43ec460cbab7b5062e7fc207, user_id=999da485e32b40e0acb68a44aa1f6821, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:43.945847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d138dc9-71ed-4f41-ac9b-b2f228598538 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:43.962166 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1304/2586] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:43 2022] PUT /identity/v3/projects/2274de7f43ec460cbab7b5062e7fc207/users/999da485e32b40e0acb68a44aa1f6821/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:43.974495 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6e0f0-552a-401e-a237-6dd8c7dc2583 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:43.974859 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6e0f0-552a-401e-a237-6dd8c7dc2583 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:43.975215 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6e0f0-552a-401e-a237-6dd8c7dc2583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:44.007788 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f8b6e0f0-552a-401e-a237-6dd8c7dc2583 None None] MFA Rules not processed for user `999da485e32b40e0acb68a44aa1f6821`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:44.009093 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f8b6e0f0-552a-401e-a237-6dd8c7dc2583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:44.018753 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:44.089523 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1283/2587] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:44.095136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d138dc9-71ed-4f41-ac9b-b2f228598538 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:44.161335 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d138dc9-71ed-4f41-ac9b-b2f228598538 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:44.166416 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d98377-b0cf-4f4e-8cf1-b00be45403e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:44.167069 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d98377-b0cf-4f4e-8cf1-b00be45403e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:44.167354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d98377-b0cf-4f4e-8cf1-b00be45403e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:44.167637 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d98377-b0cf-4f4e-8cf1-b00be45403e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:44.168480 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d98377-b0cf-4f4e-8cf1-b00be45403e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:44.170394 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d98377-b0cf-4f4e-8cf1-b00be45403e7 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:44.199382 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1305/2588] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:45:44 2022] POST /identity/v3/projects => generated 378 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:44.205279 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6e0f0-552a-401e-a237-6dd8c7dc2583 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:44.217882 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6e0f0-552a-401e-a237-6dd8c7dc2583 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:44.280288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24672e-6c11-4dd7-882d-16ceabb87075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:44.280929 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24672e-6c11-4dd7-882d-16ceabb87075 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:44.281267 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24672e-6c11-4dd7-882d-16ceabb87075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:44.281719 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24672e-6c11-4dd7-882d-16ceabb87075 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:44.282448 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be24672e-6c11-4dd7-882d-16ceabb87075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:44.284786 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be24672e-6c11-4dd7-882d-16ceabb87075 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:44.306907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1284/2589] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:45:44 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:44.311315 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d98377-b0cf-4f4e-8cf1-b00be45403e7 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:44.322910 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d98377-b0cf-4f4e-8cf1-b00be45403e7 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:44.328230 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43484edf-9958-4029-8c37-e870a22197a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:44.328875 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43484edf-9958-4029-8c37-e870a22197a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:44.329161 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43484edf-9958-4029-8c37-e870a22197a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:44.329438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43484edf-9958-4029-8c37-e870a22197a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:44.330156 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43484edf-9958-4029-8c37-e870a22197a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:44.331801 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43484edf-9958-4029-8c37-e870a22197a6 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:44.343512 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1306/2590] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:44 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:44.348359 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24672e-6c11-4dd7-882d-16ceabb87075 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:44.359898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24672e-6c11-4dd7-882d-16ceabb87075 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:44.365842 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a1914-0830-4c20-bd84-390cfbfb0db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:44.366524 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a1914-0830-4c20-bd84-390cfbfb0db8 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:44.366808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a1914-0830-4c20-bd84-390cfbfb0db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:44.367087 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a1914-0830-4c20-bd84-390cfbfb0db8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:44.367811 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a1914-0830-4c20-bd84-390cfbfb0db8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:44.369499 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a1914-0830-4c20-bd84-390cfbfb0db8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:44.381085 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1285/2591] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:44 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:44.387214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43484edf-9958-4029-8c37-e870a22197a6 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:44.397994 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43484edf-9958-4029-8c37-e870a22197a6 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:44.404779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd35a9b-1e3c-433c-84c1-4dedb5f8c137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:44.406174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd35a9b-1e3c-433c-84c1-4dedb5f8c137 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:44.406754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd35a9b-1e3c-433c-84c1-4dedb5f8c137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:44.407248 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd35a9b-1e3c-433c-84c1-4dedb5f8c137 admin admin] PATH_INFO: `/v3/projects/163616cca4cc4015885818cf6309e804/users/4431c0ce4e7c4e898adb48adcb08773d/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:44.427310 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd35a9b-1e3c-433c-84c1-4dedb5f8c137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=163616cca4cc4015885818cf6309e804, user_id=4431c0ce4e7c4e898adb48adcb08773d, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:44.430023 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd35a9b-1e3c-433c-84c1-4dedb5f8c137 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:44.445299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1307/2592] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:44 2022] PUT /identity/v3/projects/163616cca4cc4015885818cf6309e804/users/4431c0ce4e7c4e898adb48adcb08773d/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:44.456593 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2fdb8-a741-4814-a7d1-4cf676c79d7f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:44.456878 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2fdb8-a741-4814-a7d1-4cf676c79d7f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:44.457151 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2fdb8-a741-4814-a7d1-4cf676c79d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:44.487682 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a0f2fdb8-a741-4814-a7d1-4cf676c79d7f None None] MFA Rules not processed for user `4431c0ce4e7c4e898adb48adcb08773d`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:44.488996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a0f2fdb8-a741-4814-a7d1-4cf676c79d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:44.563071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1286/2593] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:44.568022 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd35a9b-1e3c-433c-84c1-4dedb5f8c137 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:44.636991 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd35a9b-1e3c-433c-84c1-4dedb5f8c137 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:44.642281 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86764620-00f9-40c9-a2ec-c36dcc2c11f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:44.642932 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86764620-00f9-40c9-a2ec-c36dcc2c11f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:44.643223 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86764620-00f9-40c9-a2ec-c36dcc2c11f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:44.643505 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86764620-00f9-40c9-a2ec-c36dcc2c11f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:44.644229 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86764620-00f9-40c9-a2ec-c36dcc2c11f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:44.646192 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86764620-00f9-40c9-a2ec-c36dcc2c11f8 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:44.668454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:44.677042 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1308/2594] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:45:44 2022] POST /identity/v3/projects => generated 378 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:44.681698 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2fdb8-a741-4814-a7d1-4cf676c79d7f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:44.693672 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2fdb8-a741-4814-a7d1-4cf676c79d7f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:44.710272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:44.768391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f8207-2234-4fba-9134-f3d750298622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:44.769003 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f8207-2234-4fba-9134-f3d750298622 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:44.769284 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f8207-2234-4fba-9134-f3d750298622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:44.769561 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f8207-2234-4fba-9134-f3d750298622 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:44.770251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f8207-2234-4fba-9134-f3d750298622 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:44.772255 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f8207-2234-4fba-9134-f3d750298622 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:44.800137 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1287/2595] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:45:44 2022] POST /identity/v3/users => generated 296 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:44.805218 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86764620-00f9-40c9-a2ec-c36dcc2c11f8 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:44.817209 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86764620-00f9-40c9-a2ec-c36dcc2c11f8 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:44.822646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9739aca-e03c-4412-9db0-582f7668f8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:44.823267 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9739aca-e03c-4412-9db0-582f7668f8ee admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:44.823555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9739aca-e03c-4412-9db0-582f7668f8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:44.823835 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9739aca-e03c-4412-9db0-582f7668f8ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:44.824595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9739aca-e03c-4412-9db0-582f7668f8ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:44.826237 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9739aca-e03c-4412-9db0-582f7668f8ee admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:44.837340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1309/2596] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:44 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:44.843231 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f8207-2234-4fba-9134-f3d750298622 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:44.855739 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f8207-2234-4fba-9134-f3d750298622 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:44.861866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580bc32-2e51-4547-8538-c1548dabf481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:44.862546 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580bc32-2e51-4547-8538-c1548dabf481 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:44.862826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580bc32-2e51-4547-8538-c1548dabf481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:44.863099 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580bc32-2e51-4547-8538-c1548dabf481 admin admin] PATH_INFO: `/v3/projects/3a5503a148c3446ba33d623264718dd4/users/2e11acd6e4304211a53702c3cdc78630/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:44.881465 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5580bc32-2e51-4547-8538-c1548dabf481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a5503a148c3446ba33d623264718dd4, user_id=2e11acd6e4304211a53702c3cdc78630, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:44.884200 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5580bc32-2e51-4547-8538-c1548dabf481 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:44.901026 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1288/2597] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:44 2022] PUT /identity/v3/projects/3a5503a148c3446ba33d623264718dd4/users/2e11acd6e4304211a53702c3cdc78630/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:44.906862 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9739aca-e03c-4412-9db0-582f7668f8ee admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:44.921311 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9739aca-e03c-4412-9db0-582f7668f8ee admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:44.987665 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8276a-2be6-43c2-bbf8-90ed050bf3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:44.988303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8276a-2be6-43c2-bbf8-90ed050bf3b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:44.988581 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8276a-2be6-43c2-bbf8-90ed050bf3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:44.988857 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8276a-2be6-43c2-bbf8-90ed050bf3b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:44.989571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8276a-2be6-43c2-bbf8-90ed050bf3b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:44.991205 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8276a-2be6-43c2-bbf8-90ed050bf3b8 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:45.002247 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1310/2598] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:44 2022] GET /identity/v3/roles => generated 1427 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:45.008479 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580bc32-2e51-4547-8538-c1548dabf481 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:45.020090 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580bc32-2e51-4547-8538-c1548dabf481 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:45.025808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6fd6eb-7e25-47f4-9cad-bf3e5ec92a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:45.026498 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fd6eb-7e25-47f4-9cad-bf3e5ec92a41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:45.026783 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fd6eb-7e25-47f4-9cad-bf3e5ec92a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:45.027069 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fd6eb-7e25-47f4-9cad-bf3e5ec92a41 admin admin] PATH_INFO: `/v3/projects/3a5503a148c3446ba33d623264718dd4/users/2e11acd6e4304211a53702c3cdc78630/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:45.029560 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6fd6eb-7e25-47f4-9cad-bf3e5ec92a41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a5503a148c3446ba33d623264718dd4, user_id=2e11acd6e4304211a53702c3cdc78630, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:45.032254 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6fd6eb-7e25-47f4-9cad-bf3e5ec92a41 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:45.048275 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1289/2599] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:45 2022] PUT /identity/v3/projects/3a5503a148c3446ba33d623264718dd4/users/2e11acd6e4304211a53702c3cdc78630/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:45.060130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3593d-4806-4db5-acbe-263f33e0a351 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:45.060602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3593d-4806-4db5-acbe-263f33e0a351 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:45.061009 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3593d-4806-4db5-acbe-263f33e0a351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:45.105177 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-abd3593d-4806-4db5-acbe-263f33e0a351 None None] MFA Rules not processed for user `2e11acd6e4304211a53702c3cdc78630`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:45.106487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-abd3593d-4806-4db5-acbe-263f33e0a351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:45.189791 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1311/2600] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:45 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:45.203822 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6fd6eb-7e25-47f4-9cad-bf3e5ec92a41 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:45.205867 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cd6fd6eb-7e25-47f4-9cad-bf3e5ec92a41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:45.267814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6fd6eb-7e25-47f4-9cad-bf3e5ec92a41 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:45.271687 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d8a0f-b7d6-4953-8bea-204bc3e2d50f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:45.272374 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d8a0f-b7d6-4953-8bea-204bc3e2d50f service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:45.272687 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d8a0f-b7d6-4953-8bea-204bc3e2d50f service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:45.272962 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d8a0f-b7d6-4953-8bea-204bc3e2d50f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:45.282612 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d8a0f-b7d6-4953-8bea-204bc3e2d50f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:45.284325 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d8a0f-b7d6-4953-8bea-204bc3e2d50f service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:45.294883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1290/2601] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:45:45 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:45:45.537099 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3593d-4806-4db5-acbe-263f33e0a351 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:45.538744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-abd3593d-4806-4db5-acbe-263f33e0a351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:45.553751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3593d-4806-4db5-acbe-263f33e0a351 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:45.558225 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e13194-9f9a-4e0f-9eac-68c336e89537 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:45.558887 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e13194-9f9a-4e0f-9eac-68c336e89537 service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:45.559176 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e13194-9f9a-4e0f-9eac-68c336e89537 service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:45.559455 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e13194-9f9a-4e0f-9eac-68c336e89537 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:45.623602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e13194-9f9a-4e0f-9eac-68c336e89537 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:45.625181 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e13194-9f9a-4e0f-9eac-68c336e89537 service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:45.634462 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1312/2602] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:45:45 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:45:46.680635 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d8a0f-b7d6-4953-8bea-204bc3e2d50f service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:46.682522 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6d6d8a0f-b7d6-4953-8bea-204bc3e2d50f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:46.755229 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d8a0f-b7d6-4953-8bea-204bc3e2d50f service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:46.759823 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd250ed-9c5d-4513-9821-353ad8815089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:46.760529 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd250ed-9c5d-4513-9821-353ad8815089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:46.760940 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd250ed-9c5d-4513-9821-353ad8815089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:46.761420 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd250ed-9c5d-4513-9821-353ad8815089 admin admin] PATH_INFO: `/v3/users/d1ae1d8d8b8d48438c816b4a468cb96a` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:46.762575 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd250ed-9c5d-4513-9821-353ad8815089 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1ae1d8d8b8d48438c816b4a468cb96a)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:46.764586 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd250ed-9c5d-4513-9821-353ad8815089 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:46.782162 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:46.818701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cbd250ed-9c5d-4513-9821-353ad8815089 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1ae1d8d8b8d48438c816b4a468cb96a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:46.826028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cbd250ed-9c5d-4513-9821-353ad8815089 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1ae1d8d8b8d48438c816b4a468cb96a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:46.826725 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cbd250ed-9c5d-4513-9821-353ad8815089 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1ae1d8d8b8d48438c816b4a468cb96a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:46.839834 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cbd250ed-9c5d-4513-9821-353ad8815089 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1ae1d8d8b8d48438c816b4a468cb96a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:46.840682 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cbd250ed-9c5d-4513-9821-353ad8815089 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1ae1d8d8b8d48438c816b4a468cb96a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:46.857064 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1291/2603] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:45:46 2022] DELETE /identity/v3/users/d1ae1d8d8b8d48438c816b4a468cb96a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:46.862261 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d074231-d38b-447e-88a4-83702412c5ea None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:46.862931 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d074231-d38b-447e-88a4-83702412c5ea None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:46.862931 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d074231-d38b-447e-88a4-83702412c5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:46.878149 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=100 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:46.885833 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-8d074231-d38b-447e-88a4-83702412c5ea None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:46.887146 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-8d074231-d38b-447e-88a4-83702412c5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:46.952744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1313/2604] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:46 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:46.963234 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd250ed-9c5d-4513-9821-353ad8815089 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:46.965085 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cbd250ed-9c5d-4513-9821-353ad8815089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:47.029588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd250ed-9c5d-4513-9821-353ad8815089 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:47.033496 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0a7ff-9d5f-4b10-837a-6d5de4134b48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:47.034136 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0a7ff-9d5f-4b10-837a-6d5de4134b48 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:47.034422 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0a7ff-9d5f-4b10-837a-6d5de4134b48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:47.034693 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0a7ff-9d5f-4b10-837a-6d5de4134b48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:47.046856 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0a7ff-9d5f-4b10-837a-6d5de4134b48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:47.048418 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0a7ff-9d5f-4b10-837a-6d5de4134b48 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:47.060606 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1292/2605] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:45:46 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:45:47.368491 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d074231-d38b-447e-88a4-83702412c5ea None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:47.369731 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-8d074231-d38b-447e-88a4-83702412c5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:47.386718 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d074231-d38b-447e-88a4-83702412c5ea None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:47.397216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4025b7-3c40-4d0c-9dac-9ef3720ea79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:47.398106 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4025b7-3c40-4d0c-9dac-9ef3720ea79c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:47.398392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4025b7-3c40-4d0c-9dac-9ef3720ea79c admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:47.398686 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4025b7-3c40-4d0c-9dac-9ef3720ea79c admin admin] PATH_INFO: `/v3/projects/480d3681ebe34d38b5504e064206b76d` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:47.399996 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4025b7-3c40-4d0c-9dac-9ef3720ea79c admin admin] RBAC: Authorizing `identity:delete_project(project_id=480d3681ebe34d38b5504e064206b76d)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:47.401994 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4025b7-3c40-4d0c-9dac-9ef3720ea79c admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:47.471925 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6a4025b7-3c40-4d0c-9dac-9ef3720ea79c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '480d3681ebe34d38b5504e064206b76d', 'request_id': 'req-6a4025b7-3c40-4d0c-9dac-9ef3720ea79c'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:47.482737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6a4025b7-3c40-4d0c-9dac-9ef3720ea79c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '480d3681ebe34d38b5504e064206b76d', 'request_id': 'req-6a4025b7-3c40-4d0c-9dac-9ef3720ea79c'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:47.497400 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6a4025b7-3c40-4d0c-9dac-9ef3720ea79c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '480d3681ebe34d38b5504e064206b76d', 'request_id': 'req-6a4025b7-3c40-4d0c-9dac-9ef3720ea79c'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:47.523746 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6a4025b7-3c40-4d0c-9dac-9ef3720ea79c admin admin] The token cache is being invalidate because project 480d3681ebe34d38b5504e064206b76d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:45:47.523746 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6a4025b7-3c40-4d0c-9dac-9ef3720ea79c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:47.524513 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6a4025b7-3c40-4d0c-9dac-9ef3720ea79c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:47.527571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1314/2606] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:45:47 2022] DELETE /identity/v3/projects/480d3681ebe34d38b5504e064206b76d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:45:47.533796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0a7ff-9d5f-4b10-837a-6d5de4134b48 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:47.547399 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0a7ff-9d5f-4b10-837a-6d5de4134b48 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:47.632190 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25fde7a-d882-49a9-bf81-41b2690e60c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:47.632867 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fde7a-d882-49a9-bf81-41b2690e60c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:47.633153 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fde7a-d882-49a9-bf81-41b2690e60c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:47.633453 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fde7a-d882-49a9-bf81-41b2690e60c9 admin admin] PATH_INFO: `/v3/users/430cf655dc954ba38edaf0b0a790b4bb` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:47.635118 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25fde7a-d882-49a9-bf81-41b2690e60c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=430cf655dc954ba38edaf0b0a790b4bb)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:47.637470 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25fde7a-d882-49a9-bf81-41b2690e60c9 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:47.684249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:47.695076 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e25fde7a-d882-49a9-bf81-41b2690e60c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '430cf655dc954ba38edaf0b0a790b4bb'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:47.704394 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e25fde7a-d882-49a9-bf81-41b2690e60c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '430cf655dc954ba38edaf0b0a790b4bb'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:47.705314 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e25fde7a-d882-49a9-bf81-41b2690e60c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '430cf655dc954ba38edaf0b0a790b4bb'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:47.726072 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e25fde7a-d882-49a9-bf81-41b2690e60c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '430cf655dc954ba38edaf0b0a790b4bb'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:47.726928 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e25fde7a-d882-49a9-bf81-41b2690e60c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '430cf655dc954ba38edaf0b0a790b4bb'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:47.743557 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1293/2607] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:45:47 2022] DELETE /identity/v3/users/430cf655dc954ba38edaf0b0a790b4bb => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:48.059346 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4025b7-3c40-4d0c-9dac-9ef3720ea79c admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:48.061016 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6a4025b7-3c40-4d0c-9dac-9ef3720ea79c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:48.076416 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4025b7-3c40-4d0c-9dac-9ef3720ea79c admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:48.138616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fbdcb-2fcc-4d35-8c92-c1de84d8eec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:48.139290 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fbdcb-2fcc-4d35-8c92-c1de84d8eec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:48.139573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fbdcb-2fcc-4d35-8c92-c1de84d8eec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:48.139858 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fbdcb-2fcc-4d35-8c92-c1de84d8eec1 admin admin] PATH_INFO: `/v3/projects/5f7292b523bc4d86b94ee4bf1b10f609` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:48.141166 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980fbdcb-2fcc-4d35-8c92-c1de84d8eec1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f7292b523bc4d86b94ee4bf1b10f609)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:48.143142 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980fbdcb-2fcc-4d35-8c92-c1de84d8eec1 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:48.185292 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:48.203445 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-980fbdcb-2fcc-4d35-8c92-c1de84d8eec1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f7292b523bc4d86b94ee4bf1b10f609', 'request_id': 'req-980fbdcb-2fcc-4d35-8c92-c1de84d8eec1'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:48.214227 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-980fbdcb-2fcc-4d35-8c92-c1de84d8eec1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f7292b523bc4d86b94ee4bf1b10f609', 'request_id': 'req-980fbdcb-2fcc-4d35-8c92-c1de84d8eec1'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:48.220782 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-980fbdcb-2fcc-4d35-8c92-c1de84d8eec1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f7292b523bc4d86b94ee4bf1b10f609', 'request_id': 'req-980fbdcb-2fcc-4d35-8c92-c1de84d8eec1'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:48.233120 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-980fbdcb-2fcc-4d35-8c92-c1de84d8eec1 admin admin] The token cache is being invalidate because project 5f7292b523bc4d86b94ee4bf1b10f609 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:45:48.233405 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-980fbdcb-2fcc-4d35-8c92-c1de84d8eec1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:48.234155 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-980fbdcb-2fcc-4d35-8c92-c1de84d8eec1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:48.235931 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1315/2608] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:45:48 2022] DELETE /identity/v3/projects/5f7292b523bc4d86b94ee4bf1b10f609 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:48.241418 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c327867-d9c1-40e2-895d-4f9c0ea98529 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:48.241711 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c327867-d9c1-40e2-895d-4f9c0ea98529 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:48.242088 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c327867-d9c1-40e2-895d-4f9c0ea98529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:48.264144 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-1c327867-d9c1-40e2-895d-4f9c0ea98529 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:48.265498 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1c327867-d9c1-40e2-895d-4f9c0ea98529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:48.344458 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1294/2609] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:48.354247 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa69cbb-56e4-4e5c-9437-c27f16c7cb8b None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:48.354569 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa69cbb-56e4-4e5c-9437-c27f16c7cb8b None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:48.354879 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa69cbb-56e4-4e5c-9437-c27f16c7cb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:48.379413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-bfa69cbb-56e4-4e5c-9437-c27f16c7cb8b None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:48.381361 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-bfa69cbb-56e4-4e5c-9437-c27f16c7cb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:48.396028 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1316/2610] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:48.402606 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c327867-d9c1-40e2-895d-4f9c0ea98529 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:48.413608 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c327867-d9c1-40e2-895d-4f9c0ea98529 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:48.419065 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580d570-56f5-4c29-abd8-d196c4be1e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:48.419698 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580d570-56f5-4c29-abd8-d196c4be1e4d admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:48.420019 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580d570-56f5-4c29-abd8-d196c4be1e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:48.420428 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580d570-56f5-4c29-abd8-d196c4be1e4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:48.421019 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580d570-56f5-4c29-abd8-d196c4be1e4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:45:48.421349 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580d570-56f5-4c29-abd8-d196c4be1e4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:48.423040 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580d570-56f5-4c29-abd8-d196c4be1e4d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:48.435431 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1295/2611] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:45:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:45:48.442016 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa69cbb-56e4-4e5c-9437-c27f16c7cb8b None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:48.451919 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa69cbb-56e4-4e5c-9437-c27f16c7cb8b None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:48.458708 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155f8ce-52a2-4ced-9d9d-47686617d019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:48.459353 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155f8ce-52a2-4ced-9d9d-47686617d019 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:48.459641 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155f8ce-52a2-4ced-9d9d-47686617d019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:48.459977 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155f8ce-52a2-4ced-9d9d-47686617d019 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:48.460668 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155f8ce-52a2-4ced-9d9d-47686617d019 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:48.462648 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155f8ce-52a2-4ced-9d9d-47686617d019 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:48.499761 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1317/2612] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:45:48 2022] POST /identity/v3/projects => generated 386 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:48.501139 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580d570-56f5-4c29-abd8-d196c4be1e4d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:48.512169 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580d570-56f5-4c29-abd8-d196c4be1e4d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:48.572079 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a6bdd-6d82-4f2b-ad3e-249c13f8d368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:48.572710 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a6bdd-6d82-4f2b-ad3e-249c13f8d368 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:48.572895 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a6bdd-6d82-4f2b-ad3e-249c13f8d368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:48.573196 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a6bdd-6d82-4f2b-ad3e-249c13f8d368 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:48.573885 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a6bdd-6d82-4f2b-ad3e-249c13f8d368 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:48.577170 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a6bdd-6d82-4f2b-ad3e-249c13f8d368 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:48.601926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1296/2613] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:45:48 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:45:48.606611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155f8ce-52a2-4ced-9d9d-47686617d019 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:48.618784 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155f8ce-52a2-4ced-9d9d-47686617d019 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:48.624879 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd37c3-98bd-4c47-9767-eaebc99b3e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:48.625525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd37c3-98bd-4c47-9767-eaebc99b3e2f admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:48.625796 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd37c3-98bd-4c47-9767-eaebc99b3e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:48.626067 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd37c3-98bd-4c47-9767-eaebc99b3e2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:48.626775 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd37c3-98bd-4c47-9767-eaebc99b3e2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:48.628444 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd37c3-98bd-4c47-9767-eaebc99b3e2f admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:48.639626 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1318/2614] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:48 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:48.644165 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a6bdd-6d82-4f2b-ad3e-249c13f8d368 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:48.655670 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a6bdd-6d82-4f2b-ad3e-249c13f8d368 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:48.661252 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd37c3-98bd-4c47-9767-eaebc99b3e2f admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:48.663049 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-33dd37c3-98bd-4c47-9767-eaebc99b3e2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:48.663186 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39e230-a029-45bc-8c15-b2aa6d9d7403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:48.663769 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39e230-a029-45bc-8c15-b2aa6d9d7403 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:48.664093 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39e230-a029-45bc-8c15-b2aa6d9d7403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:48.664410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39e230-a029-45bc-8c15-b2aa6d9d7403 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:48.665134 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39e230-a029-45bc-8c15-b2aa6d9d7403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:48.666783 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39e230-a029-45bc-8c15-b2aa6d9d7403 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:48.676427 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd37c3-98bd-4c47-9767-eaebc99b3e2f admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:48.681839 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1297/2615] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:48 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:48.681990 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d88f6f-f67c-43cd-b7bf-e0b920912ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:48.682236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d88f6f-f67c-43cd-b7bf-e0b920912ebb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:48.682479 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d88f6f-f67c-43cd-b7bf-e0b920912ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:48.682756 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d88f6f-f67c-43cd-b7bf-e0b920912ebb admin admin] PATH_INFO: `/v3/users/999da485e32b40e0acb68a44aa1f6821` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:48.683176 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39e230-a029-45bc-8c15-b2aa6d9d7403 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:48.684066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d88f6f-f67c-43cd-b7bf-e0b920912ebb admin admin] RBAC: Authorizing `identity:delete_user(user_id=999da485e32b40e0acb68a44aa1f6821)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:48.686021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d88f6f-f67c-43cd-b7bf-e0b920912ebb admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:48.693547 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39e230-a029-45bc-8c15-b2aa6d9d7403 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:48.699638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5478ec90-595d-4e84-a77f-3a4da3f4d47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:48.700370 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5478ec90-595d-4e84-a77f-3a4da3f4d47d admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:48.700656 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5478ec90-595d-4e84-a77f-3a4da3f4d47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:48.700936 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5478ec90-595d-4e84-a77f-3a4da3f4d47d admin admin] PATH_INFO: `/v3/projects/27ee291226e440e78d760194c9a25472/users/d57570e952b24173a86041f35baa1a51/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:48.720194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5478ec90-595d-4e84-a77f-3a4da3f4d47d admin admin] RBAC: Authorizing `identity:create_grant(project_id=27ee291226e440e78d760194c9a25472, user_id=d57570e952b24173a86041f35baa1a51, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:48.723189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5478ec90-595d-4e84-a77f-3a4da3f4d47d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:48.726585 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:48.738629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b7d88f6f-f67c-43cd-b7bf-e0b920912ebb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '999da485e32b40e0acb68a44aa1f6821'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:48.739412 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b7d88f6f-f67c-43cd-b7bf-e0b920912ebb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '999da485e32b40e0acb68a44aa1f6821'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:48.745618 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1298/2616] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:48 2022] PUT /identity/v3/projects/27ee291226e440e78d760194c9a25472/users/d57570e952b24173a86041f35baa1a51/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:45:48.751567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be31bf-7a0f-4fb4-a6a9-3021ddb952c7 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:48.751866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be31bf-7a0f-4fb4-a6a9-3021ddb952c7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:48.752180 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be31bf-7a0f-4fb4-a6a9-3021ddb952c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:48.759933 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b7d88f6f-f67c-43cd-b7bf-e0b920912ebb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '999da485e32b40e0acb68a44aa1f6821'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:48.768364 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b7d88f6f-f67c-43cd-b7bf-e0b920912ebb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '999da485e32b40e0acb68a44aa1f6821'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:48.777533 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b7d88f6f-f67c-43cd-b7bf-e0b920912ebb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '999da485e32b40e0acb68a44aa1f6821'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:48.782956 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-97be31bf-7a0f-4fb4-a6a9-3021ddb952c7 None None] MFA Rules not processed for user `d57570e952b24173a86041f35baa1a51`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:48.784307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-97be31bf-7a0f-4fb4-a6a9-3021ddb952c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:48.784799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1319/2617] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:45:48 2022] DELETE /identity/v3/users/999da485e32b40e0acb68a44aa1f6821 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:45:48.789222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4efc2d-774d-45c4-97d2-04a6fc026313 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:48.790220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4efc2d-774d-45c4-97d2-04a6fc026313 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:48.790486 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4efc2d-774d-45c4-97d2-04a6fc026313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:48.811050 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-4a4efc2d-774d-45c4-97d2-04a6fc026313 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:48.812370 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-4a4efc2d-774d-45c4-97d2-04a6fc026313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:48.855187 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1299/2618] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:48 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:48.859819 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be31bf-7a0f-4fb4-a6a9-3021ddb952c7 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:48.861249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-97be31bf-7a0f-4fb4-a6a9-3021ddb952c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:48.863634 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:48.878791 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1320/2619] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:45:48.889651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4efc2d-774d-45c4-97d2-04a6fc026313 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:48.891087 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-4a4efc2d-774d-45c4-97d2-04a6fc026313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:48.930969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be31bf-7a0f-4fb4-a6a9-3021ddb952c7 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:48.935985 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a117d8b-b7e5-4fe7-9ad3-71b33e6eef99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:48.936657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a117d8b-b7e5-4fe7-9ad3-71b33e6eef99 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:48.936916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a117d8b-b7e5-4fe7-9ad3-71b33e6eef99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:48.937186 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a117d8b-b7e5-4fe7-9ad3-71b33e6eef99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:48.937898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a117d8b-b7e5-4fe7-9ad3-71b33e6eef99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:48.939865 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a117d8b-b7e5-4fe7-9ad3-71b33e6eef99 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:48.953031 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4efc2d-774d-45c4-97d2-04a6fc026313 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:48.957012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1dfd6-045b-4b84-aac2-c2f1a9e79cfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:48.957673 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1dfd6-045b-4b84-aac2-c2f1a9e79cfa service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:48.957995 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1dfd6-045b-4b84-aac2-c2f1a9e79cfa service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:48.958255 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1dfd6-045b-4b84-aac2-c2f1a9e79cfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:48.968167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1300/2620] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:45:48 2022] POST /identity/v3/projects => generated 386 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:45:48.972125 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1dfd6-045b-4b84-aac2-c2f1a9e79cfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:48.972759 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a117d8b-b7e5-4fe7-9ad3-71b33e6eef99 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:48.973617 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1dfd6-045b-4b84-aac2-c2f1a9e79cfa service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:48.984128 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1321/2621] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:45:48 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:45:49.040469 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a117d8b-b7e5-4fe7-9ad3-71b33e6eef99 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:49.045588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c76a93-6fef-45f1-bf15-b5b753d7b04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:49.046372 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c76a93-6fef-45f1-bf15-b5b753d7b04f admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:49.046665 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c76a93-6fef-45f1-bf15-b5b753d7b04f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:49.046943 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c76a93-6fef-45f1-bf15-b5b753d7b04f admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:49.047638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c76a93-6fef-45f1-bf15-b5b753d7b04f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:49.049647 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c76a93-6fef-45f1-bf15-b5b753d7b04f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:49.067875 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:49.073485 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1301/2622] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:45:48 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:45:49.078068 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1dfd6-045b-4b84-aac2-c2f1a9e79cfa service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:49.088909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1dfd6-045b-4b84-aac2-c2f1a9e79cfa service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:49.095121 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937d107-5721-437a-a246-3d057c6607d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:49.096095 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937d107-5721-437a-a246-3d057c6607d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:49.096503 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937d107-5721-437a-a246-3d057c6607d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:49.096750 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937d107-5721-437a-a246-3d057c6607d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:49.097469 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937d107-5721-437a-a246-3d057c6607d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:49.099143 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937d107-5721-437a-a246-3d057c6607d4 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:49.110990 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1322/2623] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:49 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:49.115340 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c76a93-6fef-45f1-bf15-b5b753d7b04f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:49.126410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c76a93-6fef-45f1-bf15-b5b753d7b04f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:49.131393 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361d6d8-e606-4a68-9d46-c967af99063d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:49.132161 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361d6d8-e606-4a68-9d46-c967af99063d admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:49.132451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361d6d8-e606-4a68-9d46-c967af99063d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:49.132712 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361d6d8-e606-4a68-9d46-c967af99063d admin admin] PATH_INFO: `/v3/projects/a9186b2fc0c74096aa8fe9c523835a6c/users/79626733491a4d5baa3ac67fa07b4d8d/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:49.152901 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f361d6d8-e606-4a68-9d46-c967af99063d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9186b2fc0c74096aa8fe9c523835a6c, user_id=79626733491a4d5baa3ac67fa07b4d8d, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:49.155620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f361d6d8-e606-4a68-9d46-c967af99063d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:49.171969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1302/2624] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:49 2022] PUT /identity/v3/projects/a9186b2fc0c74096aa8fe9c523835a6c/users/79626733491a4d5baa3ac67fa07b4d8d/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:49.176292 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937d107-5721-437a-a246-3d057c6607d4 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:49.187578 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937d107-5721-437a-a246-3d057c6607d4 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:49.250908 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e7696e-2fc7-4c13-a317-8f5b417f0dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:49.251564 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7696e-2fc7-4c13-a317-8f5b417f0dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:49.251853 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7696e-2fc7-4c13-a317-8f5b417f0dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:49.252165 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7696e-2fc7-4c13-a317-8f5b417f0dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:49.252900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e7696e-2fc7-4c13-a317-8f5b417f0dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:49.254664 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e7696e-2fc7-4c13-a317-8f5b417f0dd4 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:49.269445 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1323/2625] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:49 2022] GET /identity/v3/roles => generated 1427 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:49.274081 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361d6d8-e606-4a68-9d46-c967af99063d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:49.282686 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e7696e-2fc7-4c13-a317-8f5b417f0dd4 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:49.284993 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-f9e7696e-2fc7-4c13-a317-8f5b417f0dd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:49.288092 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361d6d8-e606-4a68-9d46-c967af99063d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:49.294490 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3652371-a6b4-4a80-8337-bc3b7b5202d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:49.295237 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3652371-a6b4-4a80-8337-bc3b7b5202d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:49.300269 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3652371-a6b4-4a80-8337-bc3b7b5202d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:49.302033 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e7696e-2fc7-4c13-a317-8f5b417f0dd4 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:49.305547 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3652371-a6b4-4a80-8337-bc3b7b5202d2 admin admin] PATH_INFO: `/v3/projects/a9186b2fc0c74096aa8fe9c523835a6c/users/79626733491a4d5baa3ac67fa07b4d8d/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:49.307253 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ccbfb-7b4a-4c51-a245-8fc8b4f25d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:49.308177 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ccbfb-7b4a-4c51-a245-8fc8b4f25d23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:49.308557 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ccbfb-7b4a-4c51-a245-8fc8b4f25d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:49.308913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ccbfb-7b4a-4c51-a245-8fc8b4f25d23 admin admin] PATH_INFO: `/v3/projects/2274de7f43ec460cbab7b5062e7fc207` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:49.310242 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ccbfb-7b4a-4c51-a245-8fc8b4f25d23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2274de7f43ec460cbab7b5062e7fc207)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:49.312326 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ccbfb-7b4a-4c51-a245-8fc8b4f25d23 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:49.313369 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3652371-a6b4-4a80-8337-bc3b7b5202d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9186b2fc0c74096aa8fe9c523835a6c, user_id=79626733491a4d5baa3ac67fa07b4d8d, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:49.316132 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3652371-a6b4-4a80-8337-bc3b7b5202d2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:49.327818 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1303/2626] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:49 2022] PUT /identity/v3/projects/a9186b2fc0c74096aa8fe9c523835a6c/users/79626733491a4d5baa3ac67fa07b4d8d/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:49.339687 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155af9f-bc32-4c35-b414-4606acf4ff6a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:49.340353 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155af9f-bc32-4c35-b414-4606acf4ff6a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:49.341354 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155af9f-bc32-4c35-b414-4606acf4ff6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:49.367466 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-209ccbfb-7b4a-4c51-a245-8fc8b4f25d23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2274de7f43ec460cbab7b5062e7fc207', 'request_id': 'req-209ccbfb-7b4a-4c51-a245-8fc8b4f25d23'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:49.372350 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9155af9f-bc32-4c35-b414-4606acf4ff6a None None] MFA Rules not processed for user `79626733491a4d5baa3ac67fa07b4d8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:49.373655 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9155af9f-bc32-4c35-b414-4606acf4ff6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:49.376853 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-209ccbfb-7b4a-4c51-a245-8fc8b4f25d23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2274de7f43ec460cbab7b5062e7fc207', 'request_id': 'req-209ccbfb-7b4a-4c51-a245-8fc8b4f25d23'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:49.383286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-209ccbfb-7b4a-4c51-a245-8fc8b4f25d23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2274de7f43ec460cbab7b5062e7fc207', 'request_id': 'req-209ccbfb-7b4a-4c51-a245-8fc8b4f25d23'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:49.398469 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-209ccbfb-7b4a-4c51-a245-8fc8b4f25d23 admin admin] The token cache is being invalidate because project 2274de7f43ec460cbab7b5062e7fc207 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:45:49.398783 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-209ccbfb-7b4a-4c51-a245-8fc8b4f25d23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:49.399721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-209ccbfb-7b4a-4c51-a245-8fc8b4f25d23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:49.402228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1324/2627] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:45:49 2022] DELETE /identity/v3/projects/2274de7f43ec460cbab7b5062e7fc207 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:49.407659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ccbfb-7b4a-4c51-a245-8fc8b4f25d23 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:49.409706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-209ccbfb-7b4a-4c51-a245-8fc8b4f25d23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:49.416514 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:49.454344 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1304/2628] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:49 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:49.466337 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155af9f-bc32-4c35-b414-4606acf4ff6a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:49.467971 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9155af9f-bc32-4c35-b414-4606acf4ff6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:49.480225 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ccbfb-7b4a-4c51-a245-8fc8b4f25d23 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:49.485142 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf7371-045b-46fc-a73d-18e0f9ad330f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:49.485818 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf7371-045b-46fc-a73d-18e0f9ad330f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:49.486103 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf7371-045b-46fc-a73d-18e0f9ad330f admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:49.486384 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf7371-045b-46fc-a73d-18e0f9ad330f admin admin] PATH_INFO: `/v3/users/4431c0ce4e7c4e898adb48adcb08773d` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:49.487606 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf7371-045b-46fc-a73d-18e0f9ad330f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4431c0ce4e7c4e898adb48adcb08773d)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:49.489641 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf7371-045b-46fc-a73d-18e0f9ad330f admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:49.535016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155af9f-bc32-4c35-b414-4606acf4ff6a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:49.538881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af10b4-8bab-473b-9778-aab804e44da1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:49.539530 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af10b4-8bab-473b-9778-aab804e44da1 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:49.539816 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af10b4-8bab-473b-9778-aab804e44da1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:49.540136 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af10b4-8bab-473b-9778-aab804e44da1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:49.540683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-57bf7371-045b-46fc-a73d-18e0f9ad330f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4431c0ce4e7c4e898adb48adcb08773d'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:49.541542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-57bf7371-045b-46fc-a73d-18e0f9ad330f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4431c0ce4e7c4e898adb48adcb08773d'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:49.550494 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af10b4-8bab-473b-9778-aab804e44da1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:49.552010 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af10b4-8bab-473b-9778-aab804e44da1 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:49.559332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-57bf7371-045b-46fc-a73d-18e0f9ad330f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4431c0ce4e7c4e898adb48adcb08773d'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:49.563553 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1305/2629] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:45:49 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:45:49.568123 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-57bf7371-045b-46fc-a73d-18e0f9ad330f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4431c0ce4e7c4e898adb48adcb08773d'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:49.580360 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-57bf7371-045b-46fc-a73d-18e0f9ad330f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4431c0ce4e7c4e898adb48adcb08773d'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:49.589502 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1325/2630] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:45:49 2022] DELETE /identity/v3/users/4431c0ce4e7c4e898adb48adcb08773d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:49.636674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af10b4-8bab-473b-9778-aab804e44da1 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:49.638415 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-50af10b4-8bab-473b-9778-aab804e44da1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:49.655744 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af10b4-8bab-473b-9778-aab804e44da1 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:49.715368 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:49.730179 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3befcea7-22ce-4395-b2ff-5698f5a466eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:49.730872 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befcea7-22ce-4395-b2ff-5698f5a466eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:49.731163 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befcea7-22ce-4395-b2ff-5698f5a466eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:49.731481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befcea7-22ce-4395-b2ff-5698f5a466eb admin admin] PATH_INFO: `/v3/users/d57570e952b24173a86041f35baa1a51` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:49.732830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3befcea7-22ce-4395-b2ff-5698f5a466eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d57570e952b24173a86041f35baa1a51)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:49.734804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3befcea7-22ce-4395-b2ff-5698f5a466eb admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:49.802167 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3befcea7-22ce-4395-b2ff-5698f5a466eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd57570e952b24173a86041f35baa1a51'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:49.810380 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3befcea7-22ce-4395-b2ff-5698f5a466eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd57570e952b24173a86041f35baa1a51'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:49.811512 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3befcea7-22ce-4395-b2ff-5698f5a466eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd57570e952b24173a86041f35baa1a51'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:49.825558 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3befcea7-22ce-4395-b2ff-5698f5a466eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd57570e952b24173a86041f35baa1a51'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:49.826446 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3befcea7-22ce-4395-b2ff-5698f5a466eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd57570e952b24173a86041f35baa1a51'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:49.841479 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1306/2631] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:45:49 2022] DELETE /identity/v3/users/d57570e952b24173a86041f35baa1a51 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:49.846852 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b36570-4614-42f5-81d5-0638c0e8bff8 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:49.847189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b36570-4614-42f5-81d5-0638c0e8bff8 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:49.847625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b36570-4614-42f5-81d5-0638c0e8bff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:49.869466 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-79b36570-4614-42f5-81d5-0638c0e8bff8 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:49.871062 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-79b36570-4614-42f5-81d5-0638c0e8bff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:49.938376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3befcea7-22ce-4395-b2ff-5698f5a466eb admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:49.940056 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3befcea7-22ce-4395-b2ff-5698f5a466eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:49.957604 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1326/2632] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:49 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:49.967815 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b36570-4614-42f5-81d5-0638c0e8bff8 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:49.969915 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-79b36570-4614-42f5-81d5-0638c0e8bff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:50.013616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:50.043419 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3befcea7-22ce-4395-b2ff-5698f5a466eb admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:50.049240 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03805a24-1040-4313-b7f4-d357b35efeed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:50.050158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03805a24-1040-4313-b7f4-d357b35efeed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:50.050471 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03805a24-1040-4313-b7f4-d357b35efeed admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:50.050975 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03805a24-1040-4313-b7f4-d357b35efeed admin admin] PATH_INFO: `/v3/projects/163616cca4cc4015885818cf6309e804` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:50.053219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03805a24-1040-4313-b7f4-d357b35efeed admin admin] RBAC: Authorizing `identity:delete_project(project_id=163616cca4cc4015885818cf6309e804)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:50.055417 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03805a24-1040-4313-b7f4-d357b35efeed admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:50.056354 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b36570-4614-42f5-81d5-0638c0e8bff8 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:50.060508 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dac284-72dc-473b-a71d-4d56bba34188 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:50.061303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dac284-72dc-473b-a71d-4d56bba34188 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:50.061683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dac284-72dc-473b-a71d-4d56bba34188 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:50.062035 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dac284-72dc-473b-a71d-4d56bba34188 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:50.072652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dac284-72dc-473b-a71d-4d56bba34188 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:50.074204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dac284-72dc-473b-a71d-4d56bba34188 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:50.084871 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1327/2633] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:45:49 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:45:50.125015 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-03805a24-1040-4313-b7f4-d357b35efeed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '163616cca4cc4015885818cf6309e804', 'request_id': 'req-03805a24-1040-4313-b7f4-d357b35efeed'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:50.132144 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-03805a24-1040-4313-b7f4-d357b35efeed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '163616cca4cc4015885818cf6309e804', 'request_id': 'req-03805a24-1040-4313-b7f4-d357b35efeed'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:50.140190 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-03805a24-1040-4313-b7f4-d357b35efeed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '163616cca4cc4015885818cf6309e804', 'request_id': 'req-03805a24-1040-4313-b7f4-d357b35efeed'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:50.152174 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:50.157861 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-03805a24-1040-4313-b7f4-d357b35efeed admin admin] The token cache is being invalidate because project 163616cca4cc4015885818cf6309e804 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:45:50.158428 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-03805a24-1040-4313-b7f4-d357b35efeed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:50.159417 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-03805a24-1040-4313-b7f4-d357b35efeed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:50.164794 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1307/2634] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:45:49 2022] DELETE /identity/v3/projects/163616cca4cc4015885818cf6309e804 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:50.166283 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dac284-72dc-473b-a71d-4d56bba34188 service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:50.177239 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dac284-72dc-473b-a71d-4d56bba34188 service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:50.241099 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983049ff-40bf-494f-9ff6-03cc0abb18e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:50.241668 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983049ff-40bf-494f-9ff6-03cc0abb18e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:50.241953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983049ff-40bf-494f-9ff6-03cc0abb18e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:50.242219 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983049ff-40bf-494f-9ff6-03cc0abb18e5 admin admin] PATH_INFO: `/v3/users/2e11acd6e4304211a53702c3cdc78630` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:50.243579 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983049ff-40bf-494f-9ff6-03cc0abb18e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e11acd6e4304211a53702c3cdc78630)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:50.245648 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983049ff-40bf-494f-9ff6-03cc0abb18e5 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:50.301731 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-983049ff-40bf-494f-9ff6-03cc0abb18e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e11acd6e4304211a53702c3cdc78630'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:50.302463 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-983049ff-40bf-494f-9ff6-03cc0abb18e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e11acd6e4304211a53702c3cdc78630'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:50.315437 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-983049ff-40bf-494f-9ff6-03cc0abb18e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e11acd6e4304211a53702c3cdc78630'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:50.323426 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-983049ff-40bf-494f-9ff6-03cc0abb18e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e11acd6e4304211a53702c3cdc78630'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:50.332789 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-983049ff-40bf-494f-9ff6-03cc0abb18e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e11acd6e4304211a53702c3cdc78630'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:50.340333 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1328/2635] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:45:50 2022] DELETE /identity/v3/users/2e11acd6e4304211a53702c3cdc78630 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:50.395044 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03805a24-1040-4313-b7f4-d357b35efeed admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:50.397036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-03805a24-1040-4313-b7f4-d357b35efeed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:50.419278 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03805a24-1040-4313-b7f4-d357b35efeed admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:50.490000 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc8900-4e04-465c-a1a5-018015e975ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:50.490782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc8900-4e04-465c-a1a5-018015e975ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:50.491145 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc8900-4e04-465c-a1a5-018015e975ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:50.491501 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc8900-4e04-465c-a1a5-018015e975ea admin admin] PATH_INFO: `/v3/projects/27ee291226e440e78d760194c9a25472` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:50.492845 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc8900-4e04-465c-a1a5-018015e975ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=27ee291226e440e78d760194c9a25472)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:50.494877 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc8900-4e04-465c-a1a5-018015e975ea admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:50.549101 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-00bc8900-4e04-465c-a1a5-018015e975ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27ee291226e440e78d760194c9a25472', 'request_id': 'req-00bc8900-4e04-465c-a1a5-018015e975ea'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:50.557444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-00bc8900-4e04-465c-a1a5-018015e975ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27ee291226e440e78d760194c9a25472', 'request_id': 'req-00bc8900-4e04-465c-a1a5-018015e975ea'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:50.564413 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-00bc8900-4e04-465c-a1a5-018015e975ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27ee291226e440e78d760194c9a25472', 'request_id': 'req-00bc8900-4e04-465c-a1a5-018015e975ea'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:50.579372 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-00bc8900-4e04-465c-a1a5-018015e975ea admin admin] The token cache is being invalidate because project 27ee291226e440e78d760194c9a25472 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:45:50.579725 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-00bc8900-4e04-465c-a1a5-018015e975ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:50.580789 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-00bc8900-4e04-465c-a1a5-018015e975ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:50.584039 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1308/2636] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:45:50 2022] DELETE /identity/v3/projects/27ee291226e440e78d760194c9a25472 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:45:50.586898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983049ff-40bf-494f-9ff6-03cc0abb18e5 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:50.588418 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-983049ff-40bf-494f-9ff6-03cc0abb18e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:50.602949 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983049ff-40bf-494f-9ff6-03cc0abb18e5 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:50.651784 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:50.657902 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc8900-4e04-465c-a1a5-018015e975ea admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:50.659815 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-00bc8900-4e04-465c-a1a5-018015e975ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:50.670108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101b2d5-fcd7-4778-b857-8d402b3c2b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:50.670806 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101b2d5-fcd7-4778-b857-8d402b3c2b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:50.671093 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101b2d5-fcd7-4778-b857-8d402b3c2b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:50.671375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101b2d5-fcd7-4778-b857-8d402b3c2b70 admin admin] PATH_INFO: `/v3/users/79626733491a4d5baa3ac67fa07b4d8d` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:50.672609 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d101b2d5-fcd7-4778-b857-8d402b3c2b70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79626733491a4d5baa3ac67fa07b4d8d)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:50.674613 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d101b2d5-fcd7-4778-b857-8d402b3c2b70 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:50.736018 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-d101b2d5-fcd7-4778-b857-8d402b3c2b70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79626733491a4d5baa3ac67fa07b4d8d'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:50.736736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-d101b2d5-fcd7-4778-b857-8d402b3c2b70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79626733491a4d5baa3ac67fa07b4d8d'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:50.746271 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc8900-4e04-465c-a1a5-018015e975ea admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:50.751079 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668815e-5d85-4c8f-8147-8b71ab262e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:50.751743 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668815e-5d85-4c8f-8147-8b71ab262e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:50.752054 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668815e-5d85-4c8f-8147-8b71ab262e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:50.752205 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-d101b2d5-fcd7-4778-b857-8d402b3c2b70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79626733491a4d5baa3ac67fa07b4d8d'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:50.752315 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668815e-5d85-4c8f-8147-8b71ab262e67 admin admin] PATH_INFO: `/v3/projects/3a5503a148c3446ba33d623264718dd4` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:50.753532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7668815e-5d85-4c8f-8147-8b71ab262e67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a5503a148c3446ba33d623264718dd4)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:50.755532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7668815e-5d85-4c8f-8147-8b71ab262e67 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:50.759417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-d101b2d5-fcd7-4778-b857-8d402b3c2b70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79626733491a4d5baa3ac67fa07b4d8d'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:50.768626 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-d101b2d5-fcd7-4778-b857-8d402b3c2b70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79626733491a4d5baa3ac67fa07b4d8d'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:50.774898 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1329/2637] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:45:50 2022] DELETE /identity/v3/users/79626733491a4d5baa3ac67fa07b4d8d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:50.795192 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:50.811624 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7668815e-5d85-4c8f-8147-8b71ab262e67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a5503a148c3446ba33d623264718dd4', 'request_id': 'req-7668815e-5d85-4c8f-8147-8b71ab262e67'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:50.820465 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7668815e-5d85-4c8f-8147-8b71ab262e67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a5503a148c3446ba33d623264718dd4', 'request_id': 'req-7668815e-5d85-4c8f-8147-8b71ab262e67'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:50.826594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7668815e-5d85-4c8f-8147-8b71ab262e67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a5503a148c3446ba33d623264718dd4', 'request_id': 'req-7668815e-5d85-4c8f-8147-8b71ab262e67'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:50.840274 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7668815e-5d85-4c8f-8147-8b71ab262e67 admin admin] The token cache is being invalidate because project 3a5503a148c3446ba33d623264718dd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:45:50.840569 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7668815e-5d85-4c8f-8147-8b71ab262e67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:50.841282 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7668815e-5d85-4c8f-8147-8b71ab262e67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:50.842909 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1309/2638] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:45:50 2022] DELETE /identity/v3/projects/3a5503a148c3446ba33d623264718dd4 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:50.849675 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf2cf7-de48-433f-a3cd-e5e5fafc9679 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:50.849984 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf2cf7-de48-433f-a3cd-e5e5fafc9679 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:50.850280 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf2cf7-de48-433f-a3cd-e5e5fafc9679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:50.872149 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-13cf2cf7-de48-433f-a3cd-e5e5fafc9679 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:50.873396 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-13cf2cf7-de48-433f-a3cd-e5e5fafc9679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:50.955914 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1330/2639] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:51.031401 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38bdf84-c1d8-4418-93b8-e0a09e38aa10 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:51.031709 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38bdf84-c1d8-4418-93b8-e0a09e38aa10 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:51.031994 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38bdf84-c1d8-4418-93b8-e0a09e38aa10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:51.056488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c38bdf84-c1d8-4418-93b8-e0a09e38aa10 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:51.057785 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c38bdf84-c1d8-4418-93b8-e0a09e38aa10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:51.066352 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1310/2640] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:51.070823 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf2cf7-de48-433f-a3cd-e5e5fafc9679 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:51.082826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf2cf7-de48-433f-a3cd-e5e5fafc9679 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:51.088000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf64ed-ab84-4631-9896-8ec179343989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:51.088714 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf64ed-ab84-4631-9896-8ec179343989 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:51.089091 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf64ed-ab84-4631-9896-8ec179343989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:51.089532 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf64ed-ab84-4631-9896-8ec179343989 admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:51.090533 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf64ed-ab84-4631-9896-8ec179343989 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:45:51.090817 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf64ed-ab84-4631-9896-8ec179343989 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:51.093165 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf64ed-ab84-4631-9896-8ec179343989 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:51.102908 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38bdf84-c1d8-4418-93b8-e0a09e38aa10 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:51.104389 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c38bdf84-c1d8-4418-93b8-e0a09e38aa10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:51.107601 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1331/2641] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:45:51 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:45:51.112416 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf64ed-ab84-4631-9896-8ec179343989 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:51.124100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf64ed-ab84-4631-9896-8ec179343989 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:51.126194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38bdf84-c1d8-4418-93b8-e0a09e38aa10 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:51.130072 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa51dcd-5962-49bf-83b0-8173580da6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:51.130864 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa51dcd-5962-49bf-83b0-8173580da6eb admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:51.131253 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa51dcd-5962-49bf-83b0-8173580da6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:51.131611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa51dcd-5962-49bf-83b0-8173580da6eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:51.132198 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd327df-a3da-47f4-9dd4-f82bd280701b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:51.132508 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa51dcd-5962-49bf-83b0-8173580da6eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:51.133010 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd327df-a3da-47f4-9dd4-f82bd280701b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:51.133549 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd327df-a3da-47f4-9dd4-f82bd280701b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:51.133921 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd327df-a3da-47f4-9dd4-f82bd280701b admin admin] PATH_INFO: `/v3/projects/a9186b2fc0c74096aa8fe9c523835a6c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:51.134840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa51dcd-5962-49bf-83b0-8173580da6eb admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:51.135897 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd327df-a3da-47f4-9dd4-f82bd280701b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9186b2fc0c74096aa8fe9c523835a6c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:51.138354 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd327df-a3da-47f4-9dd4-f82bd280701b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:51.167652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1332/2642] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:45:51 2022] POST /identity/v3/projects => generated 386 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:45:51.172207 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa51dcd-5962-49bf-83b0-8173580da6eb admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:51.197955 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bbd327df-a3da-47f4-9dd4-f82bd280701b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9186b2fc0c74096aa8fe9c523835a6c', 'request_id': 'req-bbd327df-a3da-47f4-9dd4-f82bd280701b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:51.205646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bbd327df-a3da-47f4-9dd4-f82bd280701b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9186b2fc0c74096aa8fe9c523835a6c', 'request_id': 'req-bbd327df-a3da-47f4-9dd4-f82bd280701b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:51.210441 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:51.213627 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bbd327df-a3da-47f4-9dd4-f82bd280701b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9186b2fc0c74096aa8fe9c523835a6c', 'request_id': 'req-bbd327df-a3da-47f4-9dd4-f82bd280701b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:51.228579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bbd327df-a3da-47f4-9dd4-f82bd280701b admin admin] The token cache is being invalidate because project a9186b2fc0c74096aa8fe9c523835a6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:45:51.229097 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bbd327df-a3da-47f4-9dd4-f82bd280701b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:51.229706 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bbd327df-a3da-47f4-9dd4-f82bd280701b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:51.231454 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1311/2643] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:45:51 2022] DELETE /identity/v3/projects/a9186b2fc0c74096aa8fe9c523835a6c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:51.236537 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4112000-ac39-4f59-bae0-6d1ae685a2b5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:51.236918 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4112000-ac39-4f59-bae0-6d1ae685a2b5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:51.237285 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4112000-ac39-4f59-bae0-6d1ae685a2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:51.246406 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa51dcd-5962-49bf-83b0-8173580da6eb admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:51.257727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d4112000-ac39-4f59-bae0-6d1ae685a2b5 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:51.259405 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d4112000-ac39-4f59-bae0-6d1ae685a2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:51.305603 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b30fc-1499-4e54-a097-d7a083465a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:51.306259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b30fc-1499-4e54-a097-d7a083465a5a admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:51.306547 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b30fc-1499-4e54-a097-d7a083465a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:51.306828 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b30fc-1499-4e54-a097-d7a083465a5a admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:51.307510 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b30fc-1499-4e54-a097-d7a083465a5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:51.309769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b30fc-1499-4e54-a097-d7a083465a5a admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:51.324079 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1312/2644] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:45:51.334874 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca8781-d862-42a8-89d9-2737c47e43bd None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:51.335697 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca8781-d862-42a8-89d9-2737c47e43bd None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:51.336003 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca8781-d862-42a8-89d9-2737c47e43bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:51.341584 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1333/2645] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:45:51 2022] POST /identity/v3/users => generated 294 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:45:51.346173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b30fc-1499-4e54-a097-d7a083465a5a admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:51.348146 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-4a0b30fc-1499-4e54-a097-d7a083465a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:51.365011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b30fc-1499-4e54-a097-d7a083465a5a admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:51.367161 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c0ca8781-d862-42a8-89d9-2737c47e43bd None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:51.368466 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c0ca8781-d862-42a8-89d9-2737c47e43bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:51.371672 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc19ce5-9880-43f2-b66c-38c1bfab8ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:51.372460 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc19ce5-9880-43f2-b66c-38c1bfab8ebe admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:51.372744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc19ce5-9880-43f2-b66c-38c1bfab8ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:51.373020 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc19ce5-9880-43f2-b66c-38c1bfab8ebe admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:51.373745 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc19ce5-9880-43f2-b66c-38c1bfab8ebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:51.375385 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc19ce5-9880-43f2-b66c-38c1bfab8ebe admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:51.388659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1334/2647] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:51 2022] GET /identity/v3/roles => generated 1427 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:51.389434 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1313/2647] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:45:51.392303 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca8781-d862-42a8-89d9-2737c47e43bd None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:51.392629 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc19ce5-9880-43f2-b66c-38c1bfab8ebe admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:51.403359 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca8781-d862-42a8-89d9-2737c47e43bd None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:51.403620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc19ce5-9880-43f2-b66c-38c1bfab8ebe admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:51.408594 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c79e51f-ff49-48aa-b9ff-27d272996977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:51.408738 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60176ab6-253e-4338-bd9e-01d8f96bb1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:51.409199 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79e51f-ff49-48aa-b9ff-27d272996977 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:51.409306 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60176ab6-253e-4338-bd9e-01d8f96bb1d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:51.409571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79e51f-ff49-48aa-b9ff-27d272996977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:51.409719 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60176ab6-253e-4338-bd9e-01d8f96bb1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:51.409816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79e51f-ff49-48aa-b9ff-27d272996977 admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:51.410111 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60176ab6-253e-4338-bd9e-01d8f96bb1d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:51.410578 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c79e51f-ff49-48aa-b9ff-27d272996977 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:45:51.410892 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c79e51f-ff49-48aa-b9ff-27d272996977 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:51.411027 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60176ab6-253e-4338-bd9e-01d8f96bb1d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:51.412591 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c79e51f-ff49-48aa-b9ff-27d272996977 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:51.412714 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60176ab6-253e-4338-bd9e-01d8f96bb1d6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:51.423897 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1314/2648] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:51 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:51.428331 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60176ab6-253e-4338-bd9e-01d8f96bb1d6 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:51.429663 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1335/2649] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:45:51 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:45:51.434194 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c79e51f-ff49-48aa-b9ff-27d272996977 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:51.438846 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60176ab6-253e-4338-bd9e-01d8f96bb1d6 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:51.444255 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213938a3-e17e-462b-8c2e-bfdf814c291c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:51.444949 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213938a3-e17e-462b-8c2e-bfdf814c291c admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:51.445227 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213938a3-e17e-462b-8c2e-bfdf814c291c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:51.445373 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c79e51f-ff49-48aa-b9ff-27d272996977 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:51.445489 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213938a3-e17e-462b-8c2e-bfdf814c291c admin admin] PATH_INFO: `/v3/projects/553a375883574e808419a11f9b256717/users/3342051dbb5a46cf8e5ca70071b62502/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:51.450762 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f9b01-fb16-4a0d-a283-8d95626f2a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:51.451322 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f9b01-fb16-4a0d-a283-8d95626f2a27 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:51.451608 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f9b01-fb16-4a0d-a283-8d95626f2a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:51.451889 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f9b01-fb16-4a0d-a283-8d95626f2a27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:51.452604 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f9b01-fb16-4a0d-a283-8d95626f2a27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:51.454519 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f9b01-fb16-4a0d-a283-8d95626f2a27 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:51.462263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213938a3-e17e-462b-8c2e-bfdf814c291c admin admin] RBAC: Authorizing `identity:create_grant(project_id=553a375883574e808419a11f9b256717, user_id=3342051dbb5a46cf8e5ca70071b62502, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:51.465028 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213938a3-e17e-462b-8c2e-bfdf814c291c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:51.468391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:51.487732 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1315/2650] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:51 2022] PUT /identity/v3/projects/553a375883574e808419a11f9b256717/users/3342051dbb5a46cf8e5ca70071b62502/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:45:51.494537 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1336/2651] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:45:51 2022] POST /identity/v3/projects => generated 382 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:45:51.498826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213938a3-e17e-462b-8c2e-bfdf814c291c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:51.505224 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56374908-fde9-4b9c-b1f5-e60d1ee553b6 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:51.505765 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56374908-fde9-4b9c-b1f5-e60d1ee553b6 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:51.506131 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56374908-fde9-4b9c-b1f5-e60d1ee553b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:51.535589 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-56374908-fde9-4b9c-b1f5-e60d1ee553b6 None None] MFA Rules not processed for user `3342051dbb5a46cf8e5ca70071b62502`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:51.537014 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-56374908-fde9-4b9c-b1f5-e60d1ee553b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:51.567752 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213938a3-e17e-462b-8c2e-bfdf814c291c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:51.611000 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1337/2652] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:51 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:45:51.623839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56374908-fde9-4b9c-b1f5-e60d1ee553b6 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:51.625378 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-56374908-fde9-4b9c-b1f5-e60d1ee553b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:51.631064 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a716fe7-2882-4e70-be66-ccf7649a05f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:51.631716 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a716fe7-2882-4e70-be66-ccf7649a05f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:51.632045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a716fe7-2882-4e70-be66-ccf7649a05f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:51.632328 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a716fe7-2882-4e70-be66-ccf7649a05f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:51.633019 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a716fe7-2882-4e70-be66-ccf7649a05f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:51.635037 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a716fe7-2882-4e70-be66-ccf7649a05f7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:51.639459 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:51.658860 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1316/2653] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:45:51 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:51.663884 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a716fe7-2882-4e70-be66-ccf7649a05f7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:51.674696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a716fe7-2882-4e70-be66-ccf7649a05f7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:51.679556 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc689834-2672-4914-aed5-eb2d20e643b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:51.680218 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc689834-2672-4914-aed5-eb2d20e643b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:51.680501 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc689834-2672-4914-aed5-eb2d20e643b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:51.680773 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc689834-2672-4914-aed5-eb2d20e643b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:51.681476 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc689834-2672-4914-aed5-eb2d20e643b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:51.683209 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc689834-2672-4914-aed5-eb2d20e643b1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:51.690953 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56374908-fde9-4b9c-b1f5-e60d1ee553b6 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:51.694791 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1317/2654] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:51 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:45:51.694949 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f9750-c28e-4937-aee4-583d4937128b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:51.695419 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f9750-c28e-4937-aee4-583d4937128b service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:51.695723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f9750-c28e-4937-aee4-583d4937128b service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:51.696082 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f9750-c28e-4937-aee4-583d4937128b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:51.699229 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc689834-2672-4914-aed5-eb2d20e643b1 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:51.707707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f9750-c28e-4937-aee4-583d4937128b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:51.709294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f9750-c28e-4937-aee4-583d4937128b service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:51.711046 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc689834-2672-4914-aed5-eb2d20e643b1 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:51.717034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffb43d-ab95-4df4-9200-53a7e361ba82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:51.718852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffb43d-ab95-4df4-9200-53a7e361ba82 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:51.719141 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffb43d-ab95-4df4-9200-53a7e361ba82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:51.719433 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffb43d-ab95-4df4-9200-53a7e361ba82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:51.719681 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1338/2655] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:45:51 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:45:51.720191 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ffb43d-ab95-4df4-9200-53a7e361ba82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:51.721852 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ffb43d-ab95-4df4-9200-53a7e361ba82 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:51.734027 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1318/2656] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:51 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:45:51.739692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f9750-c28e-4937-aee4-583d4937128b service cinder] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:51.740044 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffb43d-ab95-4df4-9200-53a7e361ba82 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:51.750459 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffb43d-ab95-4df4-9200-53a7e361ba82 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:51.752429 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f9750-c28e-4937-aee4-583d4937128b service cinder] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:51.755251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bcd5cc-ad0e-4be6-8b53-04e4cb0ec1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:51.756118 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcd5cc-ad0e-4be6-8b53-04e4cb0ec1fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:51.756401 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcd5cc-ad0e-4be6-8b53-04e4cb0ec1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:51.756678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcd5cc-ad0e-4be6-8b53-04e4cb0ec1fe admin admin] PATH_INFO: `/v3/users/3342051dbb5a46cf8e5ca70071b62502` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:51.757387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e4f09-70a1-4196-b711-0f4177162a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:51.757870 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bcd5cc-ad0e-4be6-8b53-04e4cb0ec1fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=3342051dbb5a46cf8e5ca70071b62502)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:51.758073 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e4f09-70a1-4196-b711-0f4177162a05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:51.758359 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e4f09-70a1-4196-b711-0f4177162a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:51.758639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e4f09-70a1-4196-b711-0f4177162a05 admin admin] PATH_INFO: `/v3/projects/dc35adf0694e4a2b8cd2c2a7f3b86048/users/aa5c02255ba147e583f7edc93cc61be7/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:51.759802 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bcd5cc-ad0e-4be6-8b53-04e4cb0ec1fe admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:51.775103 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e4f09-70a1-4196-b711-0f4177162a05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc35adf0694e4a2b8cd2c2a7f3b86048, user_id=aa5c02255ba147e583f7edc93cc61be7, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:51.777909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e4f09-70a1-4196-b711-0f4177162a05 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:51.793546 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1339/2657] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:51 2022] PUT /identity/v3/projects/dc35adf0694e4a2b8cd2c2a7f3b86048/users/aa5c02255ba147e583f7edc93cc61be7/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:51.806430 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e5af2-5206-492f-bc9b-2c208dc17285 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:51.806750 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e5af2-5206-492f-bc9b-2c208dc17285 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:51.807028 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e5af2-5206-492f-bc9b-2c208dc17285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:51.809435 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a9bcd5cc-ad0e-4be6-8b53-04e4cb0ec1fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3342051dbb5a46cf8e5ca70071b62502'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:51.816905 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a9bcd5cc-ad0e-4be6-8b53-04e4cb0ec1fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3342051dbb5a46cf8e5ca70071b62502'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:51.817640 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a9bcd5cc-ad0e-4be6-8b53-04e4cb0ec1fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3342051dbb5a46cf8e5ca70071b62502'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:51.825350 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:51.830980 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a9bcd5cc-ad0e-4be6-8b53-04e4cb0ec1fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3342051dbb5a46cf8e5ca70071b62502'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:51.831712 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a9bcd5cc-ad0e-4be6-8b53-04e4cb0ec1fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3342051dbb5a46cf8e5ca70071b62502'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:51.836922 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-6a3e5af2-5206-492f-bc9b-2c208dc17285 None None] MFA Rules not processed for user `aa5c02255ba147e583f7edc93cc61be7`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:51.838402 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6a3e5af2-5206-492f-bc9b-2c208dc17285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:51.848822 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1319/2658] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:45:51 2022] DELETE /identity/v3/users/3342051dbb5a46cf8e5ca70071b62502 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:45:51.853567 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa47b8-52a2-48a0-a87c-a606f1e56ebe None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:51.853863 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa47b8-52a2-48a0-a87c-a606f1e56ebe None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:51.854147 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa47b8-52a2-48a0-a87c-a606f1e56ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:51.873741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-fcfa47b8-52a2-48a0-a87c-a606f1e56ebe None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:51.875004 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fcfa47b8-52a2-48a0-a87c-a606f1e56ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:51.909762 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1340/2659] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:45:51.914962 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e5af2-5206-492f-bc9b-2c208dc17285 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:51.916766 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6a3e5af2-5206-492f-bc9b-2c208dc17285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:51.944882 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1320/2660] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:45:51.952922 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa47b8-52a2-48a0-a87c-a606f1e56ebe None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:51.955569 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fcfa47b8-52a2-48a0-a87c-a606f1e56ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:51.985840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e5af2-5206-492f-bc9b-2c208dc17285 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:51.990783 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6590f-d1a5-4b95-b7cf-bdc84d13a81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:51.991436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6590f-d1a5-4b95-b7cf-bdc84d13a81b admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:51.991729 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6590f-d1a5-4b95-b7cf-bdc84d13a81b admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:51.992035 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6590f-d1a5-4b95-b7cf-bdc84d13a81b admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:51.992733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6590f-d1a5-4b95-b7cf-bdc84d13a81b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:51.994746 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6590f-d1a5-4b95-b7cf-bdc84d13a81b admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:52.017821 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa47b8-52a2-48a0-a87c-a606f1e56ebe None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:52.022291 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736f98f2-62fa-4bee-a002-78b549553947 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:52.022941 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f98f2-62fa-4bee-a002-78b549553947 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:52.023261 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f98f2-62fa-4bee-a002-78b549553947 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:52.023380 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1341/2661] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:45:51 2022] POST /identity/v3/projects => generated 380 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:45:52.024182 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f98f2-62fa-4bee-a002-78b549553947 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:52.027763 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6590f-d1a5-4b95-b7cf-bdc84d13a81b admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:52.034032 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:52.035618 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736f98f2-62fa-4bee-a002-78b549553947 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:52.037146 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736f98f2-62fa-4bee-a002-78b549553947 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:52.047865 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1321/2662] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:45:51 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:45:52.102851 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6590f-d1a5-4b95-b7cf-bdc84d13a81b admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:52.108868 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589b40cf-cc91-49b9-9851-f501a2a8c971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:52.109628 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b40cf-cc91-49b9-9851-f501a2a8c971 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:52.109917 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b40cf-cc91-49b9-9851-f501a2a8c971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:52.110198 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b40cf-cc91-49b9-9851-f501a2a8c971 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:52.110963 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589b40cf-cc91-49b9-9851-f501a2a8c971 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:52.113033 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589b40cf-cc91-49b9-9851-f501a2a8c971 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:52.140196 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1342/2663] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:45:52 2022] POST /identity/v3/users => generated 297 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:45:52.142069 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736f98f2-62fa-4bee-a002-78b549553947 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:52.154214 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736f98f2-62fa-4bee-a002-78b549553947 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:52.160263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349dd41a-4a77-463f-85fc-b8f7a62412d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:52.161004 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349dd41a-4a77-463f-85fc-b8f7a62412d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:52.161359 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349dd41a-4a77-463f-85fc-b8f7a62412d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:52.161642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349dd41a-4a77-463f-85fc-b8f7a62412d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:52.162397 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349dd41a-4a77-463f-85fc-b8f7a62412d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:52.164060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349dd41a-4a77-463f-85fc-b8f7a62412d5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:52.175307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1322/2664] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:52 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:52.180438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589b40cf-cc91-49b9-9851-f501a2a8c971 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:52.192315 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589b40cf-cc91-49b9-9851-f501a2a8c971 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:52.199556 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ccd8f3-7907-47e1-a38d-50eb80f91000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:52.200872 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccd8f3-7907-47e1-a38d-50eb80f91000 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:52.201357 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccd8f3-7907-47e1-a38d-50eb80f91000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:52.201665 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccd8f3-7907-47e1-a38d-50eb80f91000 admin admin] PATH_INFO: `/v3/projects/31f4f6973b3a40909ab912ada09ed4f8/users/cccbb54ffc26435182d4e9dd33c87333/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:52.223253 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ccd8f3-7907-47e1-a38d-50eb80f91000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31f4f6973b3a40909ab912ada09ed4f8, user_id=cccbb54ffc26435182d4e9dd33c87333, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:52.227140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ccd8f3-7907-47e1-a38d-50eb80f91000 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:52.242745 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1343/2665] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:52 2022] PUT /identity/v3/projects/31f4f6973b3a40909ab912ada09ed4f8/users/cccbb54ffc26435182d4e9dd33c87333/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:52.247213 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349dd41a-4a77-463f-85fc-b8f7a62412d5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:52.258393 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349dd41a-4a77-463f-85fc-b8f7a62412d5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:52.322319 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81707f02-a7e7-40c0-b5b7-945d413997c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:52.323115 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81707f02-a7e7-40c0-b5b7-945d413997c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:52.323410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81707f02-a7e7-40c0-b5b7-945d413997c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:52.323691 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81707f02-a7e7-40c0-b5b7-945d413997c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:52.324573 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81707f02-a7e7-40c0-b5b7-945d413997c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:52.326297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81707f02-a7e7-40c0-b5b7-945d413997c5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:52.338127 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1323/2666] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:52 2022] GET /identity/v3/roles => generated 1427 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:52.342024 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ccd8f3-7907-47e1-a38d-50eb80f91000 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:52.342192 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81707f02-a7e7-40c0-b5b7-945d413997c5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:52.344771 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-f0ccd8f3-7907-47e1-a38d-50eb80f91000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:52.353298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81707f02-a7e7-40c0-b5b7-945d413997c5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:52.359202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b80ceea-775a-4f83-9521-66e97269d6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:52.359904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80ceea-775a-4f83-9521-66e97269d6bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:52.360224 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80ceea-775a-4f83-9521-66e97269d6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:52.360504 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80ceea-775a-4f83-9521-66e97269d6bf admin admin] PATH_INFO: `/v3/projects/31f4f6973b3a40909ab912ada09ed4f8/users/cccbb54ffc26435182d4e9dd33c87333/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:52.361550 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ccd8f3-7907-47e1-a38d-50eb80f91000 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:52.362558 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b80ceea-775a-4f83-9521-66e97269d6bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=31f4f6973b3a40909ab912ada09ed4f8, user_id=cccbb54ffc26435182d4e9dd33c87333, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:52.365343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b80ceea-775a-4f83-9521-66e97269d6bf admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:52.367475 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c8a56-9e2f-420e-9e7d-f769b9442748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:52.368488 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c8a56-9e2f-420e-9e7d-f769b9442748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:52.368866 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c8a56-9e2f-420e-9e7d-f769b9442748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:52.369367 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c8a56-9e2f-420e-9e7d-f769b9442748 admin admin] PATH_INFO: `/v3/projects/553a375883574e808419a11f9b256717` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:52.371157 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c8a56-9e2f-420e-9e7d-f769b9442748 admin admin] RBAC: Authorizing `identity:delete_project(project_id=553a375883574e808419a11f9b256717)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:52.372266 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:52.374484 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c8a56-9e2f-420e-9e7d-f769b9442748 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:52.379796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1324/2667] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:52 2022] PUT /identity/v3/projects/31f4f6973b3a40909ab912ada09ed4f8/users/cccbb54ffc26435182d4e9dd33c87333/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:45:52.392431 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3bcb3c-3590-4ff2-b029-34e97df75009 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:52.392718 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3bcb3c-3590-4ff2-b029-34e97df75009 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:52.392998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3bcb3c-3590-4ff2-b029-34e97df75009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:52.422921 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-1f3bcb3c-3590-4ff2-b029-34e97df75009 None None] MFA Rules not processed for user `cccbb54ffc26435182d4e9dd33c87333`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:52.424218 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1f3bcb3c-3590-4ff2-b029-34e97df75009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:52.430869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-138c8a56-9e2f-420e-9e7d-f769b9442748 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '553a375883574e808419a11f9b256717', 'request_id': 'req-138c8a56-9e2f-420e-9e7d-f769b9442748'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:52.440345 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-138c8a56-9e2f-420e-9e7d-f769b9442748 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '553a375883574e808419a11f9b256717', 'request_id': 'req-138c8a56-9e2f-420e-9e7d-f769b9442748'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:52.446619 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-138c8a56-9e2f-420e-9e7d-f769b9442748 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '553a375883574e808419a11f9b256717', 'request_id': 'req-138c8a56-9e2f-420e-9e7d-f769b9442748'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:52.458129 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-138c8a56-9e2f-420e-9e7d-f769b9442748 admin admin] The token cache is being invalidate because project 553a375883574e808419a11f9b256717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:45:52.458425 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-138c8a56-9e2f-420e-9e7d-f769b9442748 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:52.459131 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-138c8a56-9e2f-420e-9e7d-f769b9442748 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:52.460809 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1344/2668] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:45:52 2022] DELETE /identity/v3/projects/553a375883574e808419a11f9b256717 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:52.466065 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc3749-e077-4382-bc85-52322987df5c None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:52.466355 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc3749-e077-4382-bc85-52322987df5c None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:52.466641 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc3749-e077-4382-bc85-52322987df5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:52.486750 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-59bc3749-e077-4382-bc85-52322987df5c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:52.488035 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-59bc3749-e077-4382-bc85-52322987df5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:52.496344 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1325/2669] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:52 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:45:52.509786 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3bcb3c-3590-4ff2-b029-34e97df75009 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:52.511337 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1f3bcb3c-3590-4ff2-b029-34e97df75009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:52.537604 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:52.552703 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1345/2670] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:45:52.561667 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9b191-7dab-4d08-b57d-c73f6d510ca5 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:52.561957 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9b191-7dab-4d08-b57d-c73f6d510ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:52.562235 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9b191-7dab-4d08-b57d-c73f6d510ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:52.581379 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3bcb3c-3590-4ff2-b029-34e97df75009 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:52.585253 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-34b9b191-7dab-4d08-b57d-c73f6d510ca5 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:52.585612 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09d563-e8b8-46ef-9409-40b273058624 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:52.586812 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09d563-e8b8-46ef-9409-40b273058624 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:52.586922 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-34b9b191-7dab-4d08-b57d-c73f6d510ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:52.587035 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09d563-e8b8-46ef-9409-40b273058624 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:52.587035 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09d563-e8b8-46ef-9409-40b273058624 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:52.597534 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09d563-e8b8-46ef-9409-40b273058624 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:52.597863 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1346/2671] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:52.599064 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09d563-e8b8-46ef-9409-40b273058624 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:52.602607 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9b191-7dab-4d08-b57d-c73f6d510ca5 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:52.609608 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1326/2672] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:45:52 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:45:52.612780 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9b191-7dab-4d08-b57d-c73f6d510ca5 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:52.617816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75013d-e831-40fc-9cbf-5058a990e15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:52.618453 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75013d-e831-40fc-9cbf-5058a990e15d admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:52.618747 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75013d-e831-40fc-9cbf-5058a990e15d admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:52.619148 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75013d-e831-40fc-9cbf-5058a990e15d admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:52.619869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75013d-e831-40fc-9cbf-5058a990e15d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:45:52.620220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75013d-e831-40fc-9cbf-5058a990e15d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:52.621851 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75013d-e831-40fc-9cbf-5058a990e15d admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:52.635203 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1347/2673] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:45:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:45:52.640031 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09d563-e8b8-46ef-9409-40b273058624 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:52.655762 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09d563-e8b8-46ef-9409-40b273058624 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:52.660683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca461ba6-73cf-4db1-9439-cc01183aaaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:52.661549 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca461ba6-73cf-4db1-9439-cc01183aaaa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:52.661704 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca461ba6-73cf-4db1-9439-cc01183aaaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:52.662026 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca461ba6-73cf-4db1-9439-cc01183aaaa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:52.662723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca461ba6-73cf-4db1-9439-cc01183aaaa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:52.664708 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca461ba6-73cf-4db1-9439-cc01183aaaa7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:52.695978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1327/2674] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:45:52 2022] POST /identity/v3/projects => generated 362 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:52.700452 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75013d-e831-40fc-9cbf-5058a990e15d admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:52.714959 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75013d-e831-40fc-9cbf-5058a990e15d admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:52.780617 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82d380-3ae9-443e-90fb-8b795d0e1dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:52.781246 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82d380-3ae9-443e-90fb-8b795d0e1dc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:52.781757 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82d380-3ae9-443e-90fb-8b795d0e1dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:52.781960 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82d380-3ae9-443e-90fb-8b795d0e1dc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:52.782595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82d380-3ae9-443e-90fb-8b795d0e1dc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:52.784589 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82d380-3ae9-443e-90fb-8b795d0e1dc6 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:52.808603 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1348/2675] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:45:52 2022] POST /identity/v3/users => generated 282 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:52.813274 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca461ba6-73cf-4db1-9439-cc01183aaaa7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:52.824297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca461ba6-73cf-4db1-9439-cc01183aaaa7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:52.829099 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3ad38-763b-4c21-aa2c-f0df3b475e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:52.829734 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3ad38-763b-4c21-aa2c-f0df3b475e03 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:52.830015 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3ad38-763b-4c21-aa2c-f0df3b475e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:52.830291 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3ad38-763b-4c21-aa2c-f0df3b475e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:52.831008 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3ad38-763b-4c21-aa2c-f0df3b475e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:52.832680 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3ad38-763b-4c21-aa2c-f0df3b475e03 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:52.844950 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1328/2676] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:52 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:45:52.849672 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82d380-3ae9-443e-90fb-8b795d0e1dc6 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:52.861532 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82d380-3ae9-443e-90fb-8b795d0e1dc6 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:52.866830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668cab01-0a10-4dac-9fba-cfdea44b0b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:52.867564 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cab01-0a10-4dac-9fba-cfdea44b0b5d admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:52.867886 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cab01-0a10-4dac-9fba-cfdea44b0b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:52.868236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cab01-0a10-4dac-9fba-cfdea44b0b5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:52.869152 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668cab01-0a10-4dac-9fba-cfdea44b0b5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:52.871115 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668cab01-0a10-4dac-9fba-cfdea44b0b5d admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:52.883736 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1349/2677] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:52 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:52.888919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3ad38-763b-4c21-aa2c-f0df3b475e03 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:52.901974 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3ad38-763b-4c21-aa2c-f0df3b475e03 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:52.907807 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7556e-0a18-4f91-afa9-e44336c8f81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:52.908778 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7556e-0a18-4f91-afa9-e44336c8f81b admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:52.909074 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7556e-0a18-4f91-afa9-e44336c8f81b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:52.909362 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7556e-0a18-4f91-afa9-e44336c8f81b admin admin] PATH_INFO: `/v3/projects/dbb003bd1c134b4d8f77d3967f4caa90/users/7308533d240645918d99e6e6ae472750/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:52.928406 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7556e-0a18-4f91-afa9-e44336c8f81b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbb003bd1c134b4d8f77d3967f4caa90, user_id=7308533d240645918d99e6e6ae472750, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:52.931109 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7556e-0a18-4f91-afa9-e44336c8f81b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:52.946746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1329/2678] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:52 2022] PUT /identity/v3/projects/dbb003bd1c134b4d8f77d3967f4caa90/users/7308533d240645918d99e6e6ae472750/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:52.959372 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34336a-0241-4633-aa27-6f793eccd9e5 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:52.959769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34336a-0241-4633-aa27-6f793eccd9e5 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:52.960174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34336a-0241-4633-aa27-6f793eccd9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:52.989364 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-0f34336a-0241-4633-aa27-6f793eccd9e5 None None] MFA Rules not processed for user `7308533d240645918d99e6e6ae472750`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:52.995547 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-0f34336a-0241-4633-aa27-6f793eccd9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:52.997970 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7556e-0a18-4f91-afa9-e44336c8f81b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:53.077189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1350/2679] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:53.115972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34336a-0241-4633-aa27-6f793eccd9e5 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:53.118034 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-0f34336a-0241-4633-aa27-6f793eccd9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:53.169799 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:53.184676 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:53.199012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34336a-0241-4633-aa27-6f793eccd9e5 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:53.203008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d8eb6-fc9e-45a9-99ac-0ccd0527794d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:53.203684 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d8eb6-fc9e-45a9-99ac-0ccd0527794d service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:53.204008 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d8eb6-fc9e-45a9-99ac-0ccd0527794d service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:53.204277 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d8eb6-fc9e-45a9-99ac-0ccd0527794d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:53.207203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7556e-0a18-4f91-afa9-e44336c8f81b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:53.212338 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573c1a29-1990-4ef1-bae4-f40aed0fc1ff tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cccbb54ffc26435182d4e9dd33c87333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f4f6973b3a40909ab912ada09ed4f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:53.212981 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c1a29-1990-4ef1-bae4-f40aed0fc1ff tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:53.213248 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d8eb6-fc9e-45a9-99ac-0ccd0527794d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:53.213396 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c1a29-1990-4ef1-bae4-f40aed0fc1ff tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:53.213586 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c1a29-1990-4ef1-bae4-f40aed0fc1ff tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:53.214693 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573c1a29-1990-4ef1-bae4-f40aed0fc1ff tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:45:53.215257 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d8eb6-fc9e-45a9-99ac-0ccd0527794d service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:53.215646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573c1a29-1990-4ef1-bae4-f40aed0fc1ff tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:53.217643 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573c1a29-1990-4ef1-bae4-f40aed0fc1ff tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:53.227178 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1351/2680] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:45:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:45:53.236393 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1330/2681] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:45:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:45:53.238832 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d8eb6-fc9e-45a9-99ac-0ccd0527794d service cinder] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:53.248994 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573c1a29-1990-4ef1-bae4-f40aed0fc1ff tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:53.251837 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d8eb6-fc9e-45a9-99ac-0ccd0527794d service cinder] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:53.257272 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b4853-3861-4d58-a7ed-43c864e2f975 tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cccbb54ffc26435182d4e9dd33c87333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f4f6973b3a40909ab912ada09ed4f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:53.257932 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b4853-3861-4d58-a7ed-43c864e2f975 tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:53.258234 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b4853-3861-4d58-a7ed-43c864e2f975 tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:53.258523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b4853-3861-4d58-a7ed-43c864e2f975 tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:53.259220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b4853-3861-4d58-a7ed-43c864e2f975 tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:53.261619 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b4853-3861-4d58-a7ed-43c864e2f975 tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:53.298515 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1352/2682] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:45:53 2022] POST /identity/v3/projects => generated 355 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:53.302945 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b4853-3861-4d58-a7ed-43c864e2f975 tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:53.327667 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573c1a29-1990-4ef1-bae4-f40aed0fc1ff tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:53.367439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b4853-3861-4d58-a7ed-43c864e2f975 tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:53.373337 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca28491-89ac-47b0-80ce-74b9d0dff2e7 tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cccbb54ffc26435182d4e9dd33c87333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f4f6973b3a40909ab912ada09ed4f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:53.373990 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca28491-89ac-47b0-80ce-74b9d0dff2e7 tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:53.374289 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca28491-89ac-47b0-80ce-74b9d0dff2e7 tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:53.374575 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca28491-89ac-47b0-80ce-74b9d0dff2e7 tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:53.375336 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca28491-89ac-47b0-80ce-74b9d0dff2e7 tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:45:53.375649 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca28491-89ac-47b0-80ce-74b9d0dff2e7 tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:53.377640 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca28491-89ac-47b0-80ce-74b9d0dff2e7 tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:53.387378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295d242-649b-4ce2-92ef-890be56189ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:53.388077 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295d242-649b-4ce2-92ef-890be56189ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:53.388366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295d242-649b-4ce2-92ef-890be56189ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:53.388643 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295d242-649b-4ce2-92ef-890be56189ea admin admin] PATH_INFO: `/v3/users/7308533d240645918d99e6e6ae472750` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:53.389943 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295d242-649b-4ce2-92ef-890be56189ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=7308533d240645918d99e6e6ae472750)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:53.390240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1353/2683] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:45:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:45:53.394178 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295d242-649b-4ce2-92ef-890be56189ea admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:53.430012 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca28491-89ac-47b0-80ce-74b9d0dff2e7 tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:53.443141 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca28491-89ac-47b0-80ce-74b9d0dff2e7 tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:53.448270 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1a80c-a017-4f72-b215-13e8b3a5de2a tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cccbb54ffc26435182d4e9dd33c87333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f4f6973b3a40909ab912ada09ed4f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:53.449058 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1a80c-a017-4f72-b215-13e8b3a5de2a tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:53.449349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1a80c-a017-4f72-b215-13e8b3a5de2a tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:53.449635 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1a80c-a017-4f72-b215-13e8b3a5de2a tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] PATH_INFO: `/v3/projects/1fc30ef6484048608643ccc66575bafe` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:53.450808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3295d242-649b-4ce2-92ef-890be56189ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7308533d240645918d99e6e6ae472750'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:53.450952 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1a80c-a017-4f72-b215-13e8b3a5de2a tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1fc30ef6484048608643ccc66575bafe)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:53.452857 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1a80c-a017-4f72-b215-13e8b3a5de2a tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:53.458705 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3295d242-649b-4ce2-92ef-890be56189ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7308533d240645918d99e6e6ae472750'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:53.459877 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3295d242-649b-4ce2-92ef-890be56189ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7308533d240645918d99e6e6ae472750'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:53.472645 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:53.473908 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3295d242-649b-4ce2-92ef-890be56189ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7308533d240645918d99e6e6ae472750'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:53.474780 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3295d242-649b-4ce2-92ef-890be56189ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7308533d240645918d99e6e6ae472750'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:53.491715 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1331/2684] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:45:53 2022] DELETE /identity/v3/users/7308533d240645918d99e6e6ae472750 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:53.496177 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47602e3-010e-4f94-a15d-38a09c01b74e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:53.496479 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47602e3-010e-4f94-a15d-38a09c01b74e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:53.496769 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47602e3-010e-4f94-a15d-38a09c01b74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:53.505302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c3a1a80c-a017-4f72-b215-13e8b3a5de2a tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fc30ef6484048608643ccc66575bafe', 'request_id': 'req-c3a1a80c-a017-4f72-b215-13e8b3a5de2a'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:53.514113 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c3a1a80c-a017-4f72-b215-13e8b3a5de2a tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fc30ef6484048608643ccc66575bafe', 'request_id': 'req-c3a1a80c-a017-4f72-b215-13e8b3a5de2a'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:53.520079 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c3a1a80c-a017-4f72-b215-13e8b3a5de2a tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fc30ef6484048608643ccc66575bafe', 'request_id': 'req-c3a1a80c-a017-4f72-b215-13e8b3a5de2a'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:53.521993 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a47602e3-010e-4f94-a15d-38a09c01b74e None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:53.523270 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a47602e3-010e-4f94-a15d-38a09c01b74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:53.534831 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c3a1a80c-a017-4f72-b215-13e8b3a5de2a tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] The token cache is being invalidate because project 1fc30ef6484048608643ccc66575bafe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:45:53.535457 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c3a1a80c-a017-4f72-b215-13e8b3a5de2a tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:53.536358 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c3a1a80c-a017-4f72-b215-13e8b3a5de2a tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:53.538516 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1354/2685] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:45:53 2022] DELETE /identity/v3/projects/1fc30ef6484048608643ccc66575bafe => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:45:53.588058 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1332/2686] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:45:53.599154 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1a80c-a017-4f72-b215-13e8b3a5de2a tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:53.600678 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c3a1a80c-a017-4f72-b215-13e8b3a5de2a tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:53.654441 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:53.678055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1a80c-a017-4f72-b215-13e8b3a5de2a tempest-VolumeQuotaClassesTest-609306450 tempest-VolumeQuotaClassesTest-609306450-project-admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:53.682437 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940bf94-8ee2-4e62-aa85-24baea74b121 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:53.683324 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940bf94-8ee2-4e62-aa85-24baea74b121 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:53.684093 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940bf94-8ee2-4e62-aa85-24baea74b121 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:53.684389 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940bf94-8ee2-4e62-aa85-24baea74b121 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:53.695714 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940bf94-8ee2-4e62-aa85-24baea74b121 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:53.697500 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940bf94-8ee2-4e62-aa85-24baea74b121 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:53.709649 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1355/2687] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:45:53 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:45:53.783660 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47602e3-010e-4f94-a15d-38a09c01b74e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:53.785510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a47602e3-010e-4f94-a15d-38a09c01b74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:53.799150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47602e3-010e-4f94-a15d-38a09c01b74e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:53.805235 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c058c1-bb13-460a-bfa5-99f86a2c32f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:53.805908 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c058c1-bb13-460a-bfa5-99f86a2c32f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:53.806223 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c058c1-bb13-460a-bfa5-99f86a2c32f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:53.806452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c058c1-bb13-460a-bfa5-99f86a2c32f2 admin admin] PATH_INFO: `/v3/users/aa5c02255ba147e583f7edc93cc61be7` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:53.807906 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c058c1-bb13-460a-bfa5-99f86a2c32f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa5c02255ba147e583f7edc93cc61be7)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:53.810044 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c058c1-bb13-460a-bfa5-99f86a2c32f2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:53.866224 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-38c058c1-bb13-460a-bfa5-99f86a2c32f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa5c02255ba147e583f7edc93cc61be7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:53.874054 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-38c058c1-bb13-460a-bfa5-99f86a2c32f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa5c02255ba147e583f7edc93cc61be7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:53.874812 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-38c058c1-bb13-460a-bfa5-99f86a2c32f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa5c02255ba147e583f7edc93cc61be7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:53.891542 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-38c058c1-bb13-460a-bfa5-99f86a2c32f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa5c02255ba147e583f7edc93cc61be7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:53.892471 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-38c058c1-bb13-460a-bfa5-99f86a2c32f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa5c02255ba147e583f7edc93cc61be7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:53.912207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1333/2688] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:45:53 2022] DELETE /identity/v3/users/aa5c02255ba147e583f7edc93cc61be7 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:53.917744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a6acc-8380-4ed7-92e2-1c5298fb2a9b None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:53.918216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a6acc-8380-4ed7-92e2-1c5298fb2a9b None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:53.918716 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a6acc-8380-4ed7-92e2-1c5298fb2a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:53.940864 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-4e0a6acc-8380-4ed7-92e2-1c5298fb2a9b None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:53.942202 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-4e0a6acc-8380-4ed7-92e2-1c5298fb2a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:54.009796 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1356/2689] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:54.022421 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c058c1-bb13-460a-bfa5-99f86a2c32f2 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:54.024120 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-38c058c1-bb13-460a-bfa5-99f86a2c32f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:54.030282 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a6acc-8380-4ed7-92e2-1c5298fb2a9b None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:54.032231 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-4e0a6acc-8380-4ed7-92e2-1c5298fb2a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:54.049405 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a6acc-8380-4ed7-92e2-1c5298fb2a9b None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:54.055158 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329dcaac-b463-4387-9e67-4c3698f714a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:54.055852 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329dcaac-b463-4387-9e67-4c3698f714a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:54.056185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329dcaac-b463-4387-9e67-4c3698f714a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:54.057518 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329dcaac-b463-4387-9e67-4c3698f714a3 admin admin] PATH_INFO: `/v3/projects/dbb003bd1c134b4d8f77d3967f4caa90` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:54.058613 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329dcaac-b463-4387-9e67-4c3698f714a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbb003bd1c134b4d8f77d3967f4caa90)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:54.061218 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329dcaac-b463-4387-9e67-4c3698f714a3 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:54.092112 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c058c1-bb13-460a-bfa5-99f86a2c32f2 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:54.097368 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8f002-60d3-4323-bc19-6464094c404c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:54.098110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8f002-60d3-4323-bc19-6464094c404c service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:54.098392 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8f002-60d3-4323-bc19-6464094c404c service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:54.098674 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8f002-60d3-4323-bc19-6464094c404c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:54.109638 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f8f002-60d3-4323-bc19-6464094c404c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:54.111131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f8f002-60d3-4323-bc19-6464094c404c service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:54.115920 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-329dcaac-b463-4387-9e67-4c3698f714a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbb003bd1c134b4d8f77d3967f4caa90', 'request_id': 'req-329dcaac-b463-4387-9e67-4c3698f714a3'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:54.122333 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1334/2690] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:45:54 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:45:54.126460 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-329dcaac-b463-4387-9e67-4c3698f714a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbb003bd1c134b4d8f77d3967f4caa90', 'request_id': 'req-329dcaac-b463-4387-9e67-4c3698f714a3'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:54.132794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-329dcaac-b463-4387-9e67-4c3698f714a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbb003bd1c134b4d8f77d3967f4caa90', 'request_id': 'req-329dcaac-b463-4387-9e67-4c3698f714a3'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:54.145672 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-329dcaac-b463-4387-9e67-4c3698f714a3 admin admin] The token cache is being invalidate because project dbb003bd1c134b4d8f77d3967f4caa90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:45:54.145978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-329dcaac-b463-4387-9e67-4c3698f714a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:54.146683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-329dcaac-b463-4387-9e67-4c3698f714a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:54.148385 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1357/2691] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:45:54 2022] DELETE /identity/v3/projects/dbb003bd1c134b4d8f77d3967f4caa90 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:54.457105 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8f002-60d3-4323-bc19-6464094c404c service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:54.458798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-76f8f002-60d3-4323-bc19-6464094c404c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:54.476687 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:54.481686 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8f002-60d3-4323-bc19-6464094c404c service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:54.546667 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a95d1c3-1c9d-43d0-a662-2223c1ce062b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:54.547321 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95d1c3-1c9d-43d0-a662-2223c1ce062b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:54.547605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95d1c3-1c9d-43d0-a662-2223c1ce062b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:54.547879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95d1c3-1c9d-43d0-a662-2223c1ce062b admin admin] PATH_INFO: `/v3/projects/dc35adf0694e4a2b8cd2c2a7f3b86048` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:54.549172 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a95d1c3-1c9d-43d0-a662-2223c1ce062b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc35adf0694e4a2b8cd2c2a7f3b86048)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:54.551087 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a95d1c3-1c9d-43d0-a662-2223c1ce062b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:54.604283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8a95d1c3-1c9d-43d0-a662-2223c1ce062b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc35adf0694e4a2b8cd2c2a7f3b86048', 'request_id': 'req-8a95d1c3-1c9d-43d0-a662-2223c1ce062b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:54.611331 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8a95d1c3-1c9d-43d0-a662-2223c1ce062b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc35adf0694e4a2b8cd2c2a7f3b86048', 'request_id': 'req-8a95d1c3-1c9d-43d0-a662-2223c1ce062b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:54.617842 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8a95d1c3-1c9d-43d0-a662-2223c1ce062b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc35adf0694e4a2b8cd2c2a7f3b86048', 'request_id': 'req-8a95d1c3-1c9d-43d0-a662-2223c1ce062b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:54.641813 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8a95d1c3-1c9d-43d0-a662-2223c1ce062b admin admin] The token cache is being invalidate because project dc35adf0694e4a2b8cd2c2a7f3b86048 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:45:54.642276 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8a95d1c3-1c9d-43d0-a662-2223c1ce062b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:54.643335 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8a95d1c3-1c9d-43d0-a662-2223c1ce062b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:54.645918 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1335/2692] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:45:54 2022] DELETE /identity/v3/projects/dc35adf0694e4a2b8cd2c2a7f3b86048 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:54.652485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329dcaac-b463-4387-9e67-4c3698f714a3 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:54.654083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-329dcaac-b463-4387-9e67-4c3698f714a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:54.668964 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329dcaac-b463-4387-9e67-4c3698f714a3 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:54.728644 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f901dcb-5ed4-4b3c-bddc-dacc8187a7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:54.729308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f901dcb-5ed4-4b3c-bddc-dacc8187a7a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:54.729652 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f901dcb-5ed4-4b3c-bddc-dacc8187a7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:54.730010 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f901dcb-5ed4-4b3c-bddc-dacc8187a7a1 admin admin] PATH_INFO: `/v3/users/cccbb54ffc26435182d4e9dd33c87333` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:54.731371 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f901dcb-5ed4-4b3c-bddc-dacc8187a7a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cccbb54ffc26435182d4e9dd33c87333)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:54.733371 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f901dcb-5ed4-4b3c-bddc-dacc8187a7a1 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:54.739534 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:54.787292 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-3f901dcb-5ed4-4b3c-bddc-dacc8187a7a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cccbb54ffc26435182d4e9dd33c87333'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:54.787978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-3f901dcb-5ed4-4b3c-bddc-dacc8187a7a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cccbb54ffc26435182d4e9dd33c87333'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:54.801579 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-3f901dcb-5ed4-4b3c-bddc-dacc8187a7a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cccbb54ffc26435182d4e9dd33c87333'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:54.809870 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-3f901dcb-5ed4-4b3c-bddc-dacc8187a7a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cccbb54ffc26435182d4e9dd33c87333'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:54.819477 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-3f901dcb-5ed4-4b3c-bddc-dacc8187a7a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cccbb54ffc26435182d4e9dd33c87333'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:54.826787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1358/2693] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:45:54 2022] DELETE /identity/v3/users/cccbb54ffc26435182d4e9dd33c87333 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:55.137583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a95d1c3-1c9d-43d0-a662-2223c1ce062b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:55.139245 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8a95d1c3-1c9d-43d0-a662-2223c1ce062b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:55.153681 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a95d1c3-1c9d-43d0-a662-2223c1ce062b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:55.225446 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d92cc-59fd-403a-8587-11740796785f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:55.226044 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d92cc-59fd-403a-8587-11740796785f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:55.226328 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d92cc-59fd-403a-8587-11740796785f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:55.226611 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d92cc-59fd-403a-8587-11740796785f admin admin] PATH_INFO: `/v3/projects/31f4f6973b3a40909ab912ada09ed4f8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:55.228248 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d92cc-59fd-403a-8587-11740796785f admin admin] RBAC: Authorizing `identity:delete_project(project_id=31f4f6973b3a40909ab912ada09ed4f8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:55.230988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d92cc-59fd-403a-8587-11740796785f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:55.278671 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:55.292132 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0e4d92cc-59fd-403a-8587-11740796785f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31f4f6973b3a40909ab912ada09ed4f8', 'request_id': 'req-0e4d92cc-59fd-403a-8587-11740796785f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:55.298536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0e4d92cc-59fd-403a-8587-11740796785f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31f4f6973b3a40909ab912ada09ed4f8', 'request_id': 'req-0e4d92cc-59fd-403a-8587-11740796785f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:55.304664 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0e4d92cc-59fd-403a-8587-11740796785f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31f4f6973b3a40909ab912ada09ed4f8', 'request_id': 'req-0e4d92cc-59fd-403a-8587-11740796785f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:55.321362 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0e4d92cc-59fd-403a-8587-11740796785f admin admin] The token cache is being invalidate because project 31f4f6973b3a40909ab912ada09ed4f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:45:55.321608 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0e4d92cc-59fd-403a-8587-11740796785f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:55.322347 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0e4d92cc-59fd-403a-8587-11740796785f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:45:55.324154 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1336/2694] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:45:55 2022] DELETE /identity/v3/projects/31f4f6973b3a40909ab912ada09ed4f8 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:55.329538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cb5ad-4cd0-45f2-9fb4-548dd9735f2c None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:55.330070 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cb5ad-4cd0-45f2-9fb4-548dd9735f2c None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:55.330175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cb5ad-4cd0-45f2-9fb4-548dd9735f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:55.351184 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-7b0cb5ad-4cd0-45f2-9fb4-548dd9735f2c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:55.352506 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-7b0cb5ad-4cd0-45f2-9fb4-548dd9735f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:55.423262 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1359/2695] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:55 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:55.431879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27fc87-44fe-4667-8590-428336a98f29 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:55.432201 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27fc87-44fe-4667-8590-428336a98f29 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:55.432484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27fc87-44fe-4667-8590-428336a98f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:55.453238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2f27fc87-44fe-4667-8590-428336a98f29 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:55.454497 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2f27fc87-44fe-4667-8590-428336a98f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:55.463092 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1337/2696] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:55 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:55.470254 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0cb5ad-4cd0-45f2-9fb4-548dd9735f2c None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:55.487813 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0cb5ad-4cd0-45f2-9fb4-548dd9735f2c None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:55.494879 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9f41c-1b67-4758-93cd-f241aed9a32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:55.495588 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9f41c-1b67-4758-93cd-f241aed9a32c admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:55.495880 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9f41c-1b67-4758-93cd-f241aed9a32c admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:55.496195 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9f41c-1b67-4758-93cd-f241aed9a32c admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:55.496923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9f41c-1b67-4758-93cd-f241aed9a32c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:45:55.497238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9f41c-1b67-4758-93cd-f241aed9a32c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:55.498888 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9f41c-1b67-4758-93cd-f241aed9a32c admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:55.512720 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1360/2697] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:45:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:45:55.518180 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27fc87-44fe-4667-8590-428336a98f29 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:55.534497 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27fc87-44fe-4667-8590-428336a98f29 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:55.542500 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c71eade-b9b1-4311-830f-f5a23060c166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:55.543166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71eade-b9b1-4311-830f-f5a23060c166 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:55.543444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71eade-b9b1-4311-830f-f5a23060c166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:55.543719 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71eade-b9b1-4311-830f-f5a23060c166 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:55.544449 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c71eade-b9b1-4311-830f-f5a23060c166 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:55.546392 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c71eade-b9b1-4311-830f-f5a23060c166 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:55.579286 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1338/2698] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:45:55 2022] POST /identity/v3/projects => generated 394 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:55.583785 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9f41c-1b67-4758-93cd-f241aed9a32c admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:55.595144 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9f41c-1b67-4758-93cd-f241aed9a32c admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:55.656539 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e81fa-17c8-4af9-8a0f-bef03f19219c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:55.657184 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e81fa-17c8-4af9-8a0f-bef03f19219c admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:55.657471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e81fa-17c8-4af9-8a0f-bef03f19219c admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:55.657764 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e81fa-17c8-4af9-8a0f-bef03f19219c admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:55.658458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213e81fa-17c8-4af9-8a0f-bef03f19219c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:55.660453 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213e81fa-17c8-4af9-8a0f-bef03f19219c admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:55.684374 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1361/2699] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:45:55 2022] POST /identity/v3/users => generated 298 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:55.688955 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c71eade-b9b1-4311-830f-f5a23060c166 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:55.700351 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c71eade-b9b1-4311-830f-f5a23060c166 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:55.705806 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36936209-9e5c-435c-837a-c650eb7827fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:55.706470 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36936209-9e5c-435c-837a-c650eb7827fe admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:55.706744 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36936209-9e5c-435c-837a-c650eb7827fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:55.707012 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36936209-9e5c-435c-837a-c650eb7827fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:55.707716 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36936209-9e5c-435c-837a-c650eb7827fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:55.709386 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36936209-9e5c-435c-837a-c650eb7827fe admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:55.720574 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1339/2700] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:55 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:55.725795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e81fa-17c8-4af9-8a0f-bef03f19219c admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:55.731336 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:55.738567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e81fa-17c8-4af9-8a0f-bef03f19219c admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:55.744842 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178007f-1d1c-4a66-974b-3132bd0678e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:55.745867 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178007f-1d1c-4a66-974b-3132bd0678e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:55.746177 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178007f-1d1c-4a66-974b-3132bd0678e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:55.746507 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178007f-1d1c-4a66-974b-3132bd0678e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:55.747299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178007f-1d1c-4a66-974b-3132bd0678e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:55.749051 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178007f-1d1c-4a66-974b-3132bd0678e8 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:55.760859 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1362/2701] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:55 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:55.767234 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36936209-9e5c-435c-837a-c650eb7827fe admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:55.778148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36936209-9e5c-435c-837a-c650eb7827fe admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:55.784345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8ecdb-7e42-480d-8a78-08a865ce9821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:55.785072 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8ecdb-7e42-480d-8a78-08a865ce9821 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:55.785381 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8ecdb-7e42-480d-8a78-08a865ce9821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:55.785663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8ecdb-7e42-480d-8a78-08a865ce9821 admin admin] PATH_INFO: `/v3/projects/69de45edb2574acabb2150f09f554b7b/users/b87567f6f55f4f2a9b8c5d37364da9a5/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:55.800745 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8ecdb-7e42-480d-8a78-08a865ce9821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69de45edb2574acabb2150f09f554b7b, user_id=b87567f6f55f4f2a9b8c5d37364da9a5, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:55.803390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8ecdb-7e42-480d-8a78-08a865ce9821 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:55.820972 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1340/2702] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:55 2022] PUT /identity/v3/projects/69de45edb2574acabb2150f09f554b7b/users/b87567f6f55f4f2a9b8c5d37364da9a5/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:55.839674 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39855d40-7001-4774-bb9c-568f7014ea6c None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:55.840011 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39855d40-7001-4774-bb9c-568f7014ea6c None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:55.840307 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39855d40-7001-4774-bb9c-568f7014ea6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:55.872642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-39855d40-7001-4774-bb9c-568f7014ea6c None None] MFA Rules not processed for user `b87567f6f55f4f2a9b8c5d37364da9a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:55.873908 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-39855d40-7001-4774-bb9c-568f7014ea6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:55.948053 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1363/2703] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:55 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:55.955097 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8ecdb-7e42-480d-8a78-08a865ce9821 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:56.025098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8ecdb-7e42-480d-8a78-08a865ce9821 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:56.030166 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f88ab8-82d2-4dc1-98f1-93262e2db83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:56.030796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f88ab8-82d2-4dc1-98f1-93262e2db83c admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:56.031083 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f88ab8-82d2-4dc1-98f1-93262e2db83c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:56.031356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f88ab8-82d2-4dc1-98f1-93262e2db83c admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:56.032066 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f88ab8-82d2-4dc1-98f1-93262e2db83c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:56.033996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f88ab8-82d2-4dc1-98f1-93262e2db83c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:56.063679 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1341/2704] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:45:55 2022] POST /identity/v3/projects => generated 394 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:56.068127 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39855d40-7001-4774-bb9c-568f7014ea6c None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:56.082566 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39855d40-7001-4774-bb9c-568f7014ea6c None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:56.152198 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9574536-57a1-4b52-8aa8-c7637d1b0a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:56.152882 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9574536-57a1-4b52-8aa8-c7637d1b0a17 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:56.153168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9574536-57a1-4b52-8aa8-c7637d1b0a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:56.153443 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9574536-57a1-4b52-8aa8-c7637d1b0a17 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:56.154129 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9574536-57a1-4b52-8aa8-c7637d1b0a17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:56.156234 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9574536-57a1-4b52-8aa8-c7637d1b0a17 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:56.181108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1364/2705] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:45:56 2022] POST /identity/v3/users => generated 304 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:45:56.186449 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f88ab8-82d2-4dc1-98f1-93262e2db83c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:56.199987 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f88ab8-82d2-4dc1-98f1-93262e2db83c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:56.206475 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd3f6f-5d4c-4711-8f51-f830820f68d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:56.207392 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd3f6f-5d4c-4711-8f51-f830820f68d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:56.207838 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd3f6f-5d4c-4711-8f51-f830820f68d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:56.208497 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd3f6f-5d4c-4711-8f51-f830820f68d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:56.209626 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd3f6f-5d4c-4711-8f51-f830820f68d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:56.212104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd3f6f-5d4c-4711-8f51-f830820f68d0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:56.224968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1342/2706] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:56 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:56.230083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9574536-57a1-4b52-8aa8-c7637d1b0a17 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:56.241556 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9574536-57a1-4b52-8aa8-c7637d1b0a17 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:56.248544 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551b102-d4e2-4dbe-aeab-1fe8e3317af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:56.249251 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551b102-d4e2-4dbe-aeab-1fe8e3317af2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:56.249552 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551b102-d4e2-4dbe-aeab-1fe8e3317af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:56.249817 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551b102-d4e2-4dbe-aeab-1fe8e3317af2 admin admin] PATH_INFO: `/v3/projects/5b4839e8e5ba46e590c73637e1d4a903/users/e9f17b4898464acf9bc3655d69b30bec/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:56.266584 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1551b102-d4e2-4dbe-aeab-1fe8e3317af2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b4839e8e5ba46e590c73637e1d4a903, user_id=e9f17b4898464acf9bc3655d69b30bec, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:56.269341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1551b102-d4e2-4dbe-aeab-1fe8e3317af2 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:56.285196 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1365/2707] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:56 2022] PUT /identity/v3/projects/5b4839e8e5ba46e590c73637e1d4a903/users/e9f17b4898464acf9bc3655d69b30bec/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:56.290111 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd3f6f-5d4c-4711-8f51-f830820f68d0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:56.300835 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd3f6f-5d4c-4711-8f51-f830820f68d0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:56.340202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:56.369811 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f8346-221e-4c7e-ad10-14ed7d2e2707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:56.370451 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f8346-221e-4c7e-ad10-14ed7d2e2707 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:56.370733 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f8346-221e-4c7e-ad10-14ed7d2e2707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:56.371013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f8346-221e-4c7e-ad10-14ed7d2e2707 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:56.371729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f8346-221e-4c7e-ad10-14ed7d2e2707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:56.373440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f8346-221e-4c7e-ad10-14ed7d2e2707 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:56.383865 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1343/2708] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:45:56 2022] GET /identity/v3/roles => generated 1427 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:45:56.388897 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551b102-d4e2-4dbe-aeab-1fe8e3317af2 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:56.400280 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551b102-d4e2-4dbe-aeab-1fe8e3317af2 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:56.405499 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a9c8b-14da-4629-8794-4c4c8fee96b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:56.406186 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a9c8b-14da-4629-8794-4c4c8fee96b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:56.406469 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a9c8b-14da-4629-8794-4c4c8fee96b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:56.406744 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a9c8b-14da-4629-8794-4c4c8fee96b2 admin admin] PATH_INFO: `/v3/projects/5b4839e8e5ba46e590c73637e1d4a903/users/e9f17b4898464acf9bc3655d69b30bec/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:56.408916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a9c8b-14da-4629-8794-4c4c8fee96b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b4839e8e5ba46e590c73637e1d4a903, user_id=e9f17b4898464acf9bc3655d69b30bec, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:56.411639 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a9c8b-14da-4629-8794-4c4c8fee96b2 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:56.423962 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1366/2709] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:45:56 2022] PUT /identity/v3/projects/5b4839e8e5ba46e590c73637e1d4a903/users/e9f17b4898464acf9bc3655d69b30bec/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:45:56.497484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e3340-83be-4f6e-9a1a-d5ca40de1d41 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:56.497830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e3340-83be-4f6e-9a1a-d5ca40de1d41 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:56.498066 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e3340-83be-4f6e-9a1a-d5ca40de1d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:56.532444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ba8e3340-83be-4f6e-9a1a-d5ca40de1d41 None None] MFA Rules not processed for user `e9f17b4898464acf9bc3655d69b30bec`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:45:56.533725 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ba8e3340-83be-4f6e-9a1a-d5ca40de1d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:56.611186 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1344/2710] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:45:56 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:45:56.622996 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a9c8b-14da-4629-8794-4c4c8fee96b2 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:56.625061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-bb7a9c8b-14da-4629-8794-4c4c8fee96b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:56.666276 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:45:56.694405 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a9c8b-14da-4629-8794-4c4c8fee96b2 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:56.698561 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfb601-94ab-45a2-832b-9b3c34a11f50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:56.699204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfb601-94ab-45a2-832b-9b3c34a11f50 service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:56.699491 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfb601-94ab-45a2-832b-9b3c34a11f50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:56.699769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfb601-94ab-45a2-832b-9b3c34a11f50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:56.710387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dfb601-94ab-45a2-832b-9b3c34a11f50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:56.711877 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dfb601-94ab-45a2-832b-9b3c34a11f50 service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:56.724367 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1367/2711] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:45:56 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:45:56.777391 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e3340-83be-4f6e-9a1a-d5ca40de1d41 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:45:56.778937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ba8e3340-83be-4f6e-9a1a-d5ca40de1d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:45:56.792420 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e3340-83be-4f6e-9a1a-d5ca40de1d41 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:45:56.796926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04dc886-0e2f-4bc0-929a-548b7f2430b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:45:56.797555 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04dc886-0e2f-4bc0-929a-548b7f2430b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:45:56.797835 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04dc886-0e2f-4bc0-929a-548b7f2430b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:45:56.798107 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04dc886-0e2f-4bc0-929a-548b7f2430b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:45:56.860877 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04dc886-0e2f-4bc0-929a-548b7f2430b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:45:56.862467 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04dc886-0e2f-4bc0-929a-548b7f2430b3 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:45:56.872633 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1345/2712] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:45:56 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:46:00.298686 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfb601-94ab-45a2-832b-9b3c34a11f50 service cinder] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:46:00.372751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfb601-94ab-45a2-832b-9b3c34a11f50 service cinder] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:46:00.377879 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0705f-c9d6-47ac-9f02-549c8477269a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:46:00.378611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0705f-c9d6-47ac-9f02-549c8477269a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:00.378893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0705f-c9d6-47ac-9f02-549c8477269a admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:00.379178 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0705f-c9d6-47ac-9f02-549c8477269a admin admin] PATH_INFO: `/v3/users/b87567f6f55f4f2a9b8c5d37364da9a5` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:00.380428 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c0705f-c9d6-47ac-9f02-549c8477269a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b87567f6f55f4f2a9b8c5d37364da9a5)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:46:00.382383 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c0705f-c9d6-47ac-9f02-549c8477269a admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:46:00.438353 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b8c0705f-c9d6-47ac-9f02-549c8477269a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b87567f6f55f4f2a9b8c5d37364da9a5'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:00.439061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b8c0705f-c9d6-47ac-9f02-549c8477269a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b87567f6f55f4f2a9b8c5d37364da9a5'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:00.453574 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b8c0705f-c9d6-47ac-9f02-549c8477269a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b87567f6f55f4f2a9b8c5d37364da9a5'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:00.461061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b8c0705f-c9d6-47ac-9f02-549c8477269a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b87567f6f55f4f2a9b8c5d37364da9a5'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:00.470951 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b8c0705f-c9d6-47ac-9f02-549c8477269a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b87567f6f55f4f2a9b8c5d37364da9a5'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:00.477516 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1368/2713] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:46:00 2022] DELETE /identity/v3/users/b87567f6f55f4f2a9b8c5d37364da9a5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:46:00.482597 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32d457-50cd-494c-801a-2e1477ef0926 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:00.483143 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32d457-50cd-494c-801a-2e1477ef0926 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:00.483143 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32d457-50cd-494c-801a-2e1477ef0926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:00.505277 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-3e32d457-50cd-494c-801a-2e1477ef0926 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:46:00.506542 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3e32d457-50cd-494c-801a-2e1477ef0926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:46:00.565760 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:46:00.581978 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1346/2714] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:46:00 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:46:00.594789 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0705f-c9d6-47ac-9f02-549c8477269a admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:46:00.596595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-b8c0705f-c9d6-47ac-9f02-549c8477269a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:46:00.671266 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0705f-c9d6-47ac-9f02-549c8477269a admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:46:00.675896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de7f65-8272-4717-be98-24dc9d2b1dc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:46:00.676590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de7f65-8272-4717-be98-24dc9d2b1dc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:00.676878 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de7f65-8272-4717-be98-24dc9d2b1dc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:00.677163 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de7f65-8272-4717-be98-24dc9d2b1dc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:00.687901 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de7f65-8272-4717-be98-24dc9d2b1dc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:46:00.689468 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de7f65-8272-4717-be98-24dc9d2b1dc8 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:46:00.693520 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:46:00.702222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1369/2715] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:46:00 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:46:01.006680 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e32d457-50cd-494c-801a-2e1477ef0926 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:46:01.008661 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3e32d457-50cd-494c-801a-2e1477ef0926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:46:01.025550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e32d457-50cd-494c-801a-2e1477ef0926 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:46:01.030750 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c37e3b-de32-4a5b-ab59-0b420a6ae326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:46:01.032617 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c37e3b-de32-4a5b-ab59-0b420a6ae326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:01.032906 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c37e3b-de32-4a5b-ab59-0b420a6ae326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:01.033387 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c37e3b-de32-4a5b-ab59-0b420a6ae326 admin admin] PATH_INFO: `/v3/projects/69de45edb2574acabb2150f09f554b7b` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:01.034727 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c37e3b-de32-4a5b-ab59-0b420a6ae326 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69de45edb2574acabb2150f09f554b7b)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:46:01.036725 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c37e3b-de32-4a5b-ab59-0b420a6ae326 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:46:01.086299 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b4c37e3b-de32-4a5b-ab59-0b420a6ae326 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69de45edb2574acabb2150f09f554b7b', 'request_id': 'req-b4c37e3b-de32-4a5b-ab59-0b420a6ae326'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:01.092785 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b4c37e3b-de32-4a5b-ab59-0b420a6ae326 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69de45edb2574acabb2150f09f554b7b', 'request_id': 'req-b4c37e3b-de32-4a5b-ab59-0b420a6ae326'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:01.099732 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b4c37e3b-de32-4a5b-ab59-0b420a6ae326 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69de45edb2574acabb2150f09f554b7b', 'request_id': 'req-b4c37e3b-de32-4a5b-ab59-0b420a6ae326'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:01.116550 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b4c37e3b-de32-4a5b-ab59-0b420a6ae326 admin admin] The token cache is being invalidate because project 69de45edb2574acabb2150f09f554b7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:46:01.116839 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b4c37e3b-de32-4a5b-ab59-0b420a6ae326 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:01.117669 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b4c37e3b-de32-4a5b-ab59-0b420a6ae326 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:01.119213 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1347/2716] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:46:01 2022] DELETE /identity/v3/projects/69de45edb2574acabb2150f09f554b7b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:46:01.123691 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de7f65-8272-4717-be98-24dc9d2b1dc8 service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:46:01.133933 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de7f65-8272-4717-be98-24dc9d2b1dc8 service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:46:01.191920 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b06d7-8226-43bb-9e11-60267875b394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:46:01.192500 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b06d7-8226-43bb-9e11-60267875b394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:01.192781 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b06d7-8226-43bb-9e11-60267875b394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:01.193031 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b06d7-8226-43bb-9e11-60267875b394 admin admin] PATH_INFO: `/v3/users/e9f17b4898464acf9bc3655d69b30bec` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:01.194244 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3b06d7-8226-43bb-9e11-60267875b394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9f17b4898464acf9bc3655d69b30bec)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:46:01.196213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3b06d7-8226-43bb-9e11-60267875b394 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:46:01.247270 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-cd3b06d7-8226-43bb-9e11-60267875b394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9f17b4898464acf9bc3655d69b30bec'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:01.248042 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-cd3b06d7-8226-43bb-9e11-60267875b394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9f17b4898464acf9bc3655d69b30bec'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:01.264999 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-cd3b06d7-8226-43bb-9e11-60267875b394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9f17b4898464acf9bc3655d69b30bec'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:01.272823 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-cd3b06d7-8226-43bb-9e11-60267875b394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9f17b4898464acf9bc3655d69b30bec'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:01.283984 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-cd3b06d7-8226-43bb-9e11-60267875b394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9f17b4898464acf9bc3655d69b30bec'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:01.291751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1370/2717] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:46:01 2022] DELETE /identity/v3/users/e9f17b4898464acf9bc3655d69b30bec => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:46:01.608735 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c37e3b-de32-4a5b-ab59-0b420a6ae326 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:46:01.612114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b4c37e3b-de32-4a5b-ab59-0b420a6ae326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:46:01.626868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c37e3b-de32-4a5b-ab59-0b420a6ae326 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:46:01.688800 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632af7e5-e43d-4aea-9f36-b56b6a03dcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:46:01.689471 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632af7e5-e43d-4aea-9f36-b56b6a03dcaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:01.689758 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632af7e5-e43d-4aea-9f36-b56b6a03dcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:01.690037 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632af7e5-e43d-4aea-9f36-b56b6a03dcaf admin admin] PATH_INFO: `/v3/projects/5b4839e8e5ba46e590c73637e1d4a903` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:01.691236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632af7e5-e43d-4aea-9f36-b56b6a03dcaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b4839e8e5ba46e590c73637e1d4a903)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:46:01.693207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632af7e5-e43d-4aea-9f36-b56b6a03dcaf admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:46:01.744407 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-632af7e5-e43d-4aea-9f36-b56b6a03dcaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b4839e8e5ba46e590c73637e1d4a903', 'request_id': 'req-632af7e5-e43d-4aea-9f36-b56b6a03dcaf'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:01.751178 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-632af7e5-e43d-4aea-9f36-b56b6a03dcaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b4839e8e5ba46e590c73637e1d4a903', 'request_id': 'req-632af7e5-e43d-4aea-9f36-b56b6a03dcaf'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:01.757126 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-632af7e5-e43d-4aea-9f36-b56b6a03dcaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b4839e8e5ba46e590c73637e1d4a903', 'request_id': 'req-632af7e5-e43d-4aea-9f36-b56b6a03dcaf'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:01.775886 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-632af7e5-e43d-4aea-9f36-b56b6a03dcaf admin admin] The token cache is being invalidate because project 5b4839e8e5ba46e590c73637e1d4a903 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:46:01.776256 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-632af7e5-e43d-4aea-9f36-b56b6a03dcaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:01.777162 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-632af7e5-e43d-4aea-9f36-b56b6a03dcaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:01.779301 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1348/2718] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:46:01 2022] DELETE /identity/v3/projects/5b4839e8e5ba46e590c73637e1d4a903 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:46:01.785665 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eeaa94-96bc-428a-ac64-d64f994947da None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:01.786250 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eeaa94-96bc-428a-ac64-d64f994947da None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:01.786250 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eeaa94-96bc-428a-ac64-d64f994947da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:01.807473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-95eeaa94-96bc-428a-ac64-d64f994947da None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:46:01.808749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-95eeaa94-96bc-428a-ac64-d64f994947da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:46:01.875770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1371/2719] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:46:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:46:01.887441 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a46ac-6461-4c7b-b205-0298de8bd119 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:01.887846 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a46ac-6461-4c7b-b205-0298de8bd119 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:01.888161 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a46ac-6461-4c7b-b205-0298de8bd119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:01.910404 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0f8a46ac-6461-4c7b-b205-0298de8bd119 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:46:01.911690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0f8a46ac-6461-4c7b-b205-0298de8bd119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:46:01.920422 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1349/2720] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:46:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:46:01.925216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eeaa94-96bc-428a-ac64-d64f994947da None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:46:01.936524 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eeaa94-96bc-428a-ac64-d64f994947da None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:46:01.941861 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137cc08a-56d8-42a2-94ca-5f85cb350521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:46:01.942516 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cc08a-56d8-42a2-94ca-5f85cb350521 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:01.942881 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cc08a-56d8-42a2-94ca-5f85cb350521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:01.943174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cc08a-56d8-42a2-94ca-5f85cb350521 admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:01.943894 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137cc08a-56d8-42a2-94ca-5f85cb350521 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:46:01.944248 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137cc08a-56d8-42a2-94ca-5f85cb350521 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:46:01.945934 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137cc08a-56d8-42a2-94ca-5f85cb350521 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:46:01.958569 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1372/2721] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:46:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:46:01.964729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a46ac-6461-4c7b-b205-0298de8bd119 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:46:01.969807 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:46:01.976605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a46ac-6461-4c7b-b205-0298de8bd119 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:46:01.983569 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83754522-7617-446b-b045-70f22962c65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:46:01.984242 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83754522-7617-446b-b045-70f22962c65d admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:01.984533 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83754522-7617-446b-b045-70f22962c65d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:01.984833 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83754522-7617-446b-b045-70f22962c65d admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:01.985523 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83754522-7617-446b-b045-70f22962c65d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:46:01.987474 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83754522-7617-446b-b045-70f22962c65d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:46:02.024427 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1350/2722] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:46:01 2022] POST /identity/v3/projects => generated 366 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:46:02.030942 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137cc08a-56d8-42a2-94ca-5f85cb350521 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:46:02.041429 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:46:02.046398 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137cc08a-56d8-42a2-94ca-5f85cb350521 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:46:02.109225 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9cd445-cfcd-4c83-8e64-046b11a5acc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:46:02.109865 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9cd445-cfcd-4c83-8e64-046b11a5acc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:02.110178 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9cd445-cfcd-4c83-8e64-046b11a5acc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:02.110454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9cd445-cfcd-4c83-8e64-046b11a5acc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:02.111134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9cd445-cfcd-4c83-8e64-046b11a5acc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:46:02.113182 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9cd445-cfcd-4c83-8e64-046b11a5acc8 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:46:02.137944 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1373/2723] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:46:02 2022] POST /identity/v3/users => generated 284 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:46:02.143491 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83754522-7617-446b-b045-70f22962c65d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:46:02.155399 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83754522-7617-446b-b045-70f22962c65d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:46:02.161741 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bbb50d-0593-40e8-9e26-046b018dfe1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:46:02.162375 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bbb50d-0593-40e8-9e26-046b018dfe1d admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:02.162655 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bbb50d-0593-40e8-9e26-046b018dfe1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:02.162928 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bbb50d-0593-40e8-9e26-046b018dfe1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:02.163636 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bbb50d-0593-40e8-9e26-046b018dfe1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:46:02.165385 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bbb50d-0593-40e8-9e26-046b018dfe1d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:46:02.176770 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1351/2724] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:46:02 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:46:02.181303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9cd445-cfcd-4c83-8e64-046b11a5acc8 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:46:02.192382 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9cd445-cfcd-4c83-8e64-046b11a5acc8 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:46:02.197728 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11a0b1-c4bd-4ac5-a1d4-42c22c905a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:46:02.198364 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11a0b1-c4bd-4ac5-a1d4-42c22c905a12 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:02.198677 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11a0b1-c4bd-4ac5-a1d4-42c22c905a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:02.198958 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11a0b1-c4bd-4ac5-a1d4-42c22c905a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:02.199685 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11a0b1-c4bd-4ac5-a1d4-42c22c905a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:46:02.201513 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11a0b1-c4bd-4ac5-a1d4-42c22c905a12 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:46:02.213245 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1374/2725] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:46:02 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:46:02.218609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bbb50d-0593-40e8-9e26-046b018dfe1d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:46:02.229288 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bbb50d-0593-40e8-9e26-046b018dfe1d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:46:02.234520 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe5def-a01c-406b-ab13-23babfd5236e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:46:02.235202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe5def-a01c-406b-ab13-23babfd5236e admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:02.235484 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe5def-a01c-406b-ab13-23babfd5236e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:02.235760 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe5def-a01c-406b-ab13-23babfd5236e admin admin] PATH_INFO: `/v3/projects/84f87b55355a45e1be3b359266427b87/users/95adf3c3ea3840f1b626637a0487d817/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:02.254095 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe5def-a01c-406b-ab13-23babfd5236e admin admin] RBAC: Authorizing `identity:create_grant(project_id=84f87b55355a45e1be3b359266427b87, user_id=95adf3c3ea3840f1b626637a0487d817, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:46:02.256803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe5def-a01c-406b-ab13-23babfd5236e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:46:02.276600 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1352/2726] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:46:02 2022] PUT /identity/v3/projects/84f87b55355a45e1be3b359266427b87/users/95adf3c3ea3840f1b626637a0487d817/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:46:02.295025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567cf67-f284-4947-99bd-445c33bdedde None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:02.295440 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567cf67-f284-4947-99bd-445c33bdedde None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:02.295881 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567cf67-f284-4947-99bd-445c33bdedde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:02.330785 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-6567cf67-f284-4947-99bd-445c33bdedde None None] MFA Rules not processed for user `95adf3c3ea3840f1b626637a0487d817`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:46:02.332117 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6567cf67-f284-4947-99bd-445c33bdedde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:46:02.404300 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1375/2727] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:46:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:46:02.409945 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe5def-a01c-406b-ab13-23babfd5236e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:46:02.485228 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe5def-a01c-406b-ab13-23babfd5236e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:46:02.490449 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b55ca5-15f5-4463-aeeb-b4dd9e8c4215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:46:02.491074 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b55ca5-15f5-4463-aeeb-b4dd9e8c4215 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:02.491368 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b55ca5-15f5-4463-aeeb-b4dd9e8c4215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:02.491645 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b55ca5-15f5-4463-aeeb-b4dd9e8c4215 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:02.492381 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b55ca5-15f5-4463-aeeb-b4dd9e8c4215 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:46:02.494440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b55ca5-15f5-4463-aeeb-b4dd9e8c4215 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:46:02.525701 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1353/2728] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:46:02 2022] POST /identity/v3/projects => generated 366 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:46:02.530606 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6567cf67-f284-4947-99bd-445c33bdedde None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:46:02.542129 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6567cf67-f284-4947-99bd-445c33bdedde None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:46:02.604644 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1660a70-9fd4-4735-958e-f60dae874052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:46:02.605290 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1660a70-9fd4-4735-958e-f60dae874052 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:02.605576 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1660a70-9fd4-4735-958e-f60dae874052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:02.605853 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1660a70-9fd4-4735-958e-f60dae874052 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:02.606581 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1660a70-9fd4-4735-958e-f60dae874052 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:46:02.608557 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1660a70-9fd4-4735-958e-f60dae874052 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:46:02.632943 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1376/2729] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:46:02 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:46:02.637410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b55ca5-15f5-4463-aeeb-b4dd9e8c4215 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:46:02.649144 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b55ca5-15f5-4463-aeeb-b4dd9e8c4215 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:46:02.655108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1bb85-0463-4a4f-b1c1-dbce6688e2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:46:02.655745 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1bb85-0463-4a4f-b1c1-dbce6688e2a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:02.656068 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1bb85-0463-4a4f-b1c1-dbce6688e2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:02.656304 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1bb85-0463-4a4f-b1c1-dbce6688e2a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:02.657054 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf1bb85-0463-4a4f-b1c1-dbce6688e2a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:46:02.658699 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf1bb85-0463-4a4f-b1c1-dbce6688e2a6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:46:02.670811 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1354/2730] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:46:02 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:46:02.675977 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1660a70-9fd4-4735-958e-f60dae874052 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:46:02.684108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:46:02.692182 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1660a70-9fd4-4735-958e-f60dae874052 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:46:02.700318 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b0e5a-22dc-46e6-aa0f-c9133667f771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:46:02.701017 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b0e5a-22dc-46e6-aa0f-c9133667f771 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:02.701306 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b0e5a-22dc-46e6-aa0f-c9133667f771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:02.701583 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b0e5a-22dc-46e6-aa0f-c9133667f771 admin admin] PATH_INFO: `/v3/projects/c126dbc677784b13a0584434f6329ac5/users/3ae16f10337e429abd2554cb1d0b8472/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:02.717857 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b0e5a-22dc-46e6-aa0f-c9133667f771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c126dbc677784b13a0584434f6329ac5, user_id=3ae16f10337e429abd2554cb1d0b8472, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:46:02.721003 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b0e5a-22dc-46e6-aa0f-c9133667f771 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:46:02.737201 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1377/2731] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:46:02 2022] PUT /identity/v3/projects/c126dbc677784b13a0584434f6329ac5/users/3ae16f10337e429abd2554cb1d0b8472/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:46:02.741599 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1bb85-0463-4a4f-b1c1-dbce6688e2a6 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:46:02.752457 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1bb85-0463-4a4f-b1c1-dbce6688e2a6 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:46:02.813095 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b3b19-14d9-4b19-a42c-98e7b045aaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:46:02.813642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b3b19-14d9-4b19-a42c-98e7b045aaae admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:02.813919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b3b19-14d9-4b19-a42c-98e7b045aaae admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:02.814173 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b3b19-14d9-4b19-a42c-98e7b045aaae admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:02.814881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b3b19-14d9-4b19-a42c-98e7b045aaae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:46:02.816531 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b3b19-14d9-4b19-a42c-98e7b045aaae admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:46:02.827570 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1355/2732] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:46:02 2022] GET /identity/v3/roles => generated 1427 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:46:02.832829 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b0e5a-22dc-46e6-aa0f-c9133667f771 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:46:02.845063 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b0e5a-22dc-46e6-aa0f-c9133667f771 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:46:02.850413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1df13-c00c-40fa-9401-758b8cd2c22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:46:02.851136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1df13-c00c-40fa-9401-758b8cd2c22f admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:02.851422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1df13-c00c-40fa-9401-758b8cd2c22f admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:02.851709 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1df13-c00c-40fa-9401-758b8cd2c22f admin admin] PATH_INFO: `/v3/projects/c126dbc677784b13a0584434f6329ac5/users/3ae16f10337e429abd2554cb1d0b8472/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:02.853925 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf1df13-c00c-40fa-9401-758b8cd2c22f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c126dbc677784b13a0584434f6329ac5, user_id=3ae16f10337e429abd2554cb1d0b8472, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:46:02.856722 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf1df13-c00c-40fa-9401-758b8cd2c22f admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:46:02.867813 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1378/2733] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:46:02 2022] PUT /identity/v3/projects/c126dbc677784b13a0584434f6329ac5/users/3ae16f10337e429abd2554cb1d0b8472/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:46:02.880817 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a941b5-9565-4407-b023-da6414c4a516 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:02.881129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a941b5-9565-4407-b023-da6414c4a516 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:02.881402 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a941b5-9565-4407-b023-da6414c4a516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:02.912913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-72a941b5-9565-4407-b023-da6414c4a516 None None] MFA Rules not processed for user `3ae16f10337e429abd2554cb1d0b8472`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:46:02.914386 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-72a941b5-9565-4407-b023-da6414c4a516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:46:02.919002 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:46:02.999883 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1356/2734] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:46:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:46:03.012260 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1df13-c00c-40fa-9401-758b8cd2c22f admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:46:03.014359 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-0bf1df13-c00c-40fa-9401-758b8cd2c22f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:46:03.077861 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1df13-c00c-40fa-9401-758b8cd2c22f admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:46:03.082249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33228326-4d2c-45db-bde1-e94b80ae6a1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:46:03.082961 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33228326-4d2c-45db-bde1-e94b80ae6a1e service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:03.083250 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33228326-4d2c-45db-bde1-e94b80ae6a1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:03.083503 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33228326-4d2c-45db-bde1-e94b80ae6a1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:03.094317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33228326-4d2c-45db-bde1-e94b80ae6a1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:46:03.095792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33228326-4d2c-45db-bde1-e94b80ae6a1e service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:46:03.105604 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1379/2735] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:46:03 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:46:03.202865 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a941b5-9565-4407-b023-da6414c4a516 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:46:03.213845 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a941b5-9565-4407-b023-da6414c4a516 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:46:03.218153 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71da56-69bb-48ad-8fb6-844c212bdbf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:46:03.219129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71da56-69bb-48ad-8fb6-844c212bdbf9 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:03.219414 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71da56-69bb-48ad-8fb6-844c212bdbf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:03.219707 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71da56-69bb-48ad-8fb6-844c212bdbf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:03.278209 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71da56-69bb-48ad-8fb6-844c212bdbf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:46:03.279669 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71da56-69bb-48ad-8fb6-844c212bdbf9 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:46:03.292244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1357/2736] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:46:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:46:08.234055 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33228326-4d2c-45db-bde1-e94b80ae6a1e service cinder] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:46:08.315023 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33228326-4d2c-45db-bde1-e94b80ae6a1e service cinder] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:46:08.322873 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df709193-f29d-4e49-98de-118963fdf36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:46:08.324078 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df709193-f29d-4e49-98de-118963fdf36d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:08.324078 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df709193-f29d-4e49-98de-118963fdf36d admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:08.324369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df709193-f29d-4e49-98de-118963fdf36d admin admin] PATH_INFO: `/v3/users/95adf3c3ea3840f1b626637a0487d817` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:08.325998 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df709193-f29d-4e49-98de-118963fdf36d admin admin] RBAC: Authorizing `identity:delete_user(user_id=95adf3c3ea3840f1b626637a0487d817)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:46:08.328006 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df709193-f29d-4e49-98de-118963fdf36d admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:46:08.389993 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-df709193-f29d-4e49-98de-118963fdf36d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95adf3c3ea3840f1b626637a0487d817'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:08.390836 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-df709193-f29d-4e49-98de-118963fdf36d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95adf3c3ea3840f1b626637a0487d817'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:08.405258 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-df709193-f29d-4e49-98de-118963fdf36d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95adf3c3ea3840f1b626637a0487d817'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:08.410397 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:46:08.414537 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-df709193-f29d-4e49-98de-118963fdf36d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95adf3c3ea3840f1b626637a0487d817'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:08.425896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-df709193-f29d-4e49-98de-118963fdf36d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95adf3c3ea3840f1b626637a0487d817'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:08.433347 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1380/2737] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:46:08 2022] DELETE /identity/v3/users/95adf3c3ea3840f1b626637a0487d817 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:46:08.438248 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bf215-2cae-4aa3-a5d7-0e4804ed6d7c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:08.438801 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bf215-2cae-4aa3-a5d7-0e4804ed6d7c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:08.438801 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bf215-2cae-4aa3-a5d7-0e4804ed6d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:08.458786 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ad5bf215-2cae-4aa3-a5d7-0e4804ed6d7c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:46:08.460088 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ad5bf215-2cae-4aa3-a5d7-0e4804ed6d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:46:08.523735 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1358/2738] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:46:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:46:08.534883 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df709193-f29d-4e49-98de-118963fdf36d admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:46:08.536447 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-df709193-f29d-4e49-98de-118963fdf36d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:46:08.599125 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df709193-f29d-4e49-98de-118963fdf36d admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:46:08.603830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa88cb6-b0ee-4595-b5d4-d8d4e5534436 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:46:08.604512 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa88cb6-b0ee-4595-b5d4-d8d4e5534436 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:08.604804 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa88cb6-b0ee-4595-b5d4-d8d4e5534436 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:08.605083 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa88cb6-b0ee-4595-b5d4-d8d4e5534436 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:08.615464 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa88cb6-b0ee-4595-b5d4-d8d4e5534436 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:46:08.616977 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa88cb6-b0ee-4595-b5d4-d8d4e5534436 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:46:08.627100 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1381/2739] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:46:08 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:46:08.929215 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5bf215-2cae-4aa3-a5d7-0e4804ed6d7c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:46:08.931787 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ad5bf215-2cae-4aa3-a5d7-0e4804ed6d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:46:08.946170 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5bf215-2cae-4aa3-a5d7-0e4804ed6d7c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:46:08.951843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81dda6-06c4-4477-a9af-a5d3eb8e9d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:46:08.952568 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81dda6-06c4-4477-a9af-a5d3eb8e9d9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:08.952858 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81dda6-06c4-4477-a9af-a5d3eb8e9d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:08.953970 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81dda6-06c4-4477-a9af-a5d3eb8e9d9a admin admin] PATH_INFO: `/v3/projects/84f87b55355a45e1be3b359266427b87` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:08.955443 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81dda6-06c4-4477-a9af-a5d3eb8e9d9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=84f87b55355a45e1be3b359266427b87)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:46:08.957450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81dda6-06c4-4477-a9af-a5d3eb8e9d9a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:46:09.010580 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:46:09.023936 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5c81dda6-06c4-4477-a9af-a5d3eb8e9d9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84f87b55355a45e1be3b359266427b87', 'request_id': 'req-5c81dda6-06c4-4477-a9af-a5d3eb8e9d9a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:09.031533 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5c81dda6-06c4-4477-a9af-a5d3eb8e9d9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84f87b55355a45e1be3b359266427b87', 'request_id': 'req-5c81dda6-06c4-4477-a9af-a5d3eb8e9d9a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:09.039925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5c81dda6-06c4-4477-a9af-a5d3eb8e9d9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84f87b55355a45e1be3b359266427b87', 'request_id': 'req-5c81dda6-06c4-4477-a9af-a5d3eb8e9d9a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:09.056342 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5c81dda6-06c4-4477-a9af-a5d3eb8e9d9a admin admin] The token cache is being invalidate because project 84f87b55355a45e1be3b359266427b87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:46:09.056628 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5c81dda6-06c4-4477-a9af-a5d3eb8e9d9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:09.057324 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5c81dda6-06c4-4477-a9af-a5d3eb8e9d9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:09.059344 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1359/2740] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:46:08 2022] DELETE /identity/v3/projects/84f87b55355a45e1be3b359266427b87 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:46:09.063436 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa88cb6-b0ee-4595-b5d4-d8d4e5534436 service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:46:09.074636 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa88cb6-b0ee-4595-b5d4-d8d4e5534436 service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:46:09.136212 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea785c3-9dde-4eb9-9ea0-d2515676342a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:46:09.136894 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea785c3-9dde-4eb9-9ea0-d2515676342a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:09.137184 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea785c3-9dde-4eb9-9ea0-d2515676342a admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:09.137435 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea785c3-9dde-4eb9-9ea0-d2515676342a admin admin] PATH_INFO: `/v3/users/3ae16f10337e429abd2554cb1d0b8472` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:09.139665 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea785c3-9dde-4eb9-9ea0-d2515676342a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ae16f10337e429abd2554cb1d0b8472)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:46:09.141811 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea785c3-9dde-4eb9-9ea0-d2515676342a admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:46:09.193578 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-1ea785c3-9dde-4eb9-9ea0-d2515676342a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ae16f10337e429abd2554cb1d0b8472'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:09.194269 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-1ea785c3-9dde-4eb9-9ea0-d2515676342a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ae16f10337e429abd2554cb1d0b8472'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:09.207659 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-1ea785c3-9dde-4eb9-9ea0-d2515676342a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ae16f10337e429abd2554cb1d0b8472'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:09.214842 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-1ea785c3-9dde-4eb9-9ea0-d2515676342a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ae16f10337e429abd2554cb1d0b8472'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:09.224661 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-1ea785c3-9dde-4eb9-9ea0-d2515676342a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ae16f10337e429abd2554cb1d0b8472'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:09.231930 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1382/2741] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:46:09 2022] DELETE /identity/v3/users/3ae16f10337e429abd2554cb1d0b8472 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:46:09.528445 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81dda6-06c4-4477-a9af-a5d3eb8e9d9a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:46:09.530932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5c81dda6-06c4-4477-a9af-a5d3eb8e9d9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:46:09.548983 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81dda6-06c4-4477-a9af-a5d3eb8e9d9a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:46:09.621571 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb2111-5102-49b8-b136-1d9d348daac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:46:09.622263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb2111-5102-49b8-b136-1d9d348daac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:09.622549 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb2111-5102-49b8-b136-1d9d348daac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:09.622828 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb2111-5102-49b8-b136-1d9d348daac1 admin admin] PATH_INFO: `/v3/projects/c126dbc677784b13a0584434f6329ac5` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:09.624165 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb2111-5102-49b8-b136-1d9d348daac1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c126dbc677784b13a0584434f6329ac5)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:46:09.626160 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb2111-5102-49b8-b136-1d9d348daac1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:46:09.676371 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7bdb2111-5102-49b8-b136-1d9d348daac1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c126dbc677784b13a0584434f6329ac5', 'request_id': 'req-7bdb2111-5102-49b8-b136-1d9d348daac1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:09.682784 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7bdb2111-5102-49b8-b136-1d9d348daac1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c126dbc677784b13a0584434f6329ac5', 'request_id': 'req-7bdb2111-5102-49b8-b136-1d9d348daac1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:09.690123 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7bdb2111-5102-49b8-b136-1d9d348daac1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c126dbc677784b13a0584434f6329ac5', 'request_id': 'req-7bdb2111-5102-49b8-b136-1d9d348daac1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:09.712434 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7bdb2111-5102-49b8-b136-1d9d348daac1 admin admin] The token cache is being invalidate because project c126dbc677784b13a0584434f6329ac5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:46:09.712868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7bdb2111-5102-49b8-b136-1d9d348daac1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:09.713848 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7bdb2111-5102-49b8-b136-1d9d348daac1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:46:09.716235 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1360/2742] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:46:09 2022] DELETE /identity/v3/projects/c126dbc677784b13a0584434f6329ac5 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:46:19.227787 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=83,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:46:19.707881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=12,keystone:SELECT=62,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:46:39.250910 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577dd3a-8230-4b71-83f4-4dac1f9aaf1e None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:39.251572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577dd3a-8230-4b71-83f4-4dac1f9aaf1e None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:39.251572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577dd3a-8230-4b71-83f4-4dac1f9aaf1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:39.274258 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-1577dd3a-8230-4b71-83f4-4dac1f9aaf1e None None] MFA Rules not processed for user `f2c14f2350dd450b846db0c22c43688b`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:46:39.275521 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-1577dd3a-8230-4b71-83f4-4dac1f9aaf1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:46:39.338557 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1383/2743] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:46:39 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:46:49.330408 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=25 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:46:57.383406 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb2111-5102-49b8-b136-1d9d348daac1 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:46:57.385412 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7bdb2111-5102-49b8-b136-1d9d348daac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:46:57.464219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb2111-5102-49b8-b136-1d9d348daac1 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:46:57.470725 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e12bf-4856-4017-b771-833e2a3210dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c7cfbe911c4378857848966f4f8d98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:46:57.471375 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e12bf-4856-4017-b771-833e2a3210dd service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:46:57.471666 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e12bf-4856-4017-b771-833e2a3210dd service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:46:57.471962 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e12bf-4856-4017-b771-833e2a3210dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:46:57.473965 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6b9e12bf-4856-4017-b771-833e2a3210dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:46:57.487570 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e12bf-4856-4017-b771-833e2a3210dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:46:57.489250 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e12bf-4856-4017-b771-833e2a3210dd service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:46:57.500581 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1361/2744] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:46:57 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:47:07.495713 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=32 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:48:08.174807 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577dd3a-8230-4b71-83f4-4dac1f9aaf1e None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:48:08.176473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-1577dd3a-8230-4b71-83f4-4dac1f9aaf1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:48:08.273551 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577dd3a-8230-4b71-83f4-4dac1f9aaf1e None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:48:08.281968 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757806af-f611-4897-91f8-5f313ead3491 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:48:08.282915 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757806af-f611-4897-91f8-5f313ead3491 service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:48:08.283327 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757806af-f611-4897-91f8-5f313ead3491 service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:48:08.283733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757806af-f611-4897-91f8-5f313ead3491 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:48:08.286370 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-757806af-f611-4897-91f8-5f313ead3491 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:48:08.300533 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757806af-f611-4897-91f8-5f313ead3491 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:48:08.302429 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757806af-f611-4897-91f8-5f313ead3491 service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:48:08.316821 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1384/2745] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:48:08 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:48:10.365370 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e12bf-4856-4017-b771-833e2a3210dd service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:48:10.367101 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6b9e12bf-4856-4017-b771-833e2a3210dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:48:10.425379 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e12bf-4856-4017-b771-833e2a3210dd service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:48:10.429373 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d5a64-80e3-41f3-bffb-064e3030090a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b31c86bc9334d2b8acc35a035f90442', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:48:10.430012 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d5a64-80e3-41f3-bffb-064e3030090a service glance] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:48:10.430294 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d5a64-80e3-41f3-bffb-064e3030090a service glance] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:48:10.430563 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d5a64-80e3-41f3-bffb-064e3030090a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:48:10.432536 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e03d5a64-80e3-41f3-bffb-064e3030090a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:48:10.447232 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d5a64-80e3-41f3-bffb-064e3030090a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:48:10.448724 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d5a64-80e3-41f3-bffb-064e3030090a service glance] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:48:10.459935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1362/2746] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:48:10 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:48:10.639019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757806af-f611-4897-91f8-5f313ead3491 service nova] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:48:10.640674 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-757806af-f611-4897-91f8-5f313ead3491 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:48:10.724778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757806af-f611-4897-91f8-5f313ead3491 service nova] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:48:10.729608 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e3233-1549-4614-93ad-d513e1e3671b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:48:10.730387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e3233-1549-4614-93ad-d513e1e3671b service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:48:10.730678 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e3233-1549-4614-93ad-d513e1e3671b service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:48:10.730956 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e3233-1549-4614-93ad-d513e1e3671b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:48:10.737327 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-f65e3233-1549-4614-93ad-d513e1e3671b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:48:10.754938 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e3233-1549-4614-93ad-d513e1e3671b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:48:10.756484 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e3233-1549-4614-93ad-d513e1e3671b service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:48:10.768383 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1385/2747] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:48:10 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:48:20.454997 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=26 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:48:20.763230 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=60 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:48:28.798620 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c02bb-754c-4a97-ade4-6a820bd11eeb None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:48:28.799156 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c02bb-754c-4a97-ade4-6a820bd11eeb None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:48:28.799267 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c02bb-754c-4a97-ade4-6a820bd11eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:48:28.824506 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-1a4c02bb-754c-4a97-ade4-6a820bd11eeb None None] MFA Rules not processed for user `efe6e59228bd49bfbff25a11eb682426`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:48:28.826096 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1a4c02bb-754c-4a97-ade4-6a820bd11eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:48:28.897065 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1363/2748] 10.222.0.121 () {58 vars in 1014 bytes} [Thu Aug 4 10:48:28 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:48:28.954373 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e3233-1549-4614-93ad-d513e1e3671b service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:48:28.956245 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-f65e3233-1549-4614-93ad-d513e1e3671b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:48:29.044267 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e3233-1549-4614-93ad-d513e1e3671b service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:48:29.052090 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c09ff9-2e97-48dc-8970-f6014debd55e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e16e26784601425c9883acd91f65becc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:48:29.052781 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c09ff9-2e97-48dc-8970-f6014debd55e service swift] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:48:29.053068 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c09ff9-2e97-48dc-8970-f6014debd55e service swift] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:48:29.053351 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c09ff9-2e97-48dc-8970-f6014debd55e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:48:29.064052 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c09ff9-2e97-48dc-8970-f6014debd55e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:48:29.065790 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c09ff9-2e97-48dc-8970-f6014debd55e service swift] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:48:29.078598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1386/2749] 10.222.0.121 () {60 vars in 1389 bytes} [Thu Aug 4 10:48:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:48:38.888874 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=25 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:48:39.072658 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=32 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:48:43.661820 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c02bb-754c-4a97-ade4-6a820bd11eeb None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:48:43.663381 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1a4c02bb-754c-4a97-ade4-6a820bd11eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:48:43.676414 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c02bb-754c-4a97-ade4-6a820bd11eeb None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:48:43.680677 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04fcb15-1eab-48a6-bbe5-1d72bc0d50f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:48:43.681427 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04fcb15-1eab-48a6-bbe5-1d72bc0d50f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:48:43.681788 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04fcb15-1eab-48a6-bbe5-1d72bc0d50f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:48:43.682081 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04fcb15-1eab-48a6-bbe5-1d72bc0d50f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:48:43.684415 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a04fcb15-1eab-48a6-bbe5-1d72bc0d50f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:48:43.696874 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04fcb15-1eab-48a6-bbe5-1d72bc0d50f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:48:43.698688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04fcb15-1eab-48a6-bbe5-1d72bc0d50f2 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:48:43.709346 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1364/2750] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:48:43 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:48:43.809064 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c09ff9-2e97-48dc-8970-f6014debd55e service swift] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:48:43.820217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c09ff9-2e97-48dc-8970-f6014debd55e service swift] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:48:43.824861 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828941e-6353-4a7a-a603-bdf33f6aefd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:48:43.825497 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828941e-6353-4a7a-a603-bdf33f6aefd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:48:43.825781 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828941e-6353-4a7a-a603-bdf33f6aefd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:48:43.826058 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828941e-6353-4a7a-a603-bdf33f6aefd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:48:43.828402 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-7828941e-6353-4a7a-a603-bdf33f6aefd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:48:43.843544 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828941e-6353-4a7a-a603-bdf33f6aefd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:48:43.845217 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828941e-6353-4a7a-a603-bdf33f6aefd7 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:48:43.856878 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1387/2751] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:48:43 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:48:45.672725 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8226a-7675-466e-8b50-cccc87079631 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:48:45.673327 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8226a-7675-466e-8b50-cccc87079631 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:48:45.673327 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8226a-7675-466e-8b50-cccc87079631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:48:45.697056 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-86b8226a-7675-466e-8b50-cccc87079631 None None] MFA Rules not processed for user `efe6e59228bd49bfbff25a11eb682426`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:48:45.698188 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-86b8226a-7675-466e-8b50-cccc87079631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:48:45.705920 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1365/2752] 10.222.0.121 () {58 vars in 1014 bytes} [Thu Aug 4 10:48:45 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:48:45.720183 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828941e-6353-4a7a-a603-bdf33f6aefd7 service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:48:45.732895 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828941e-6353-4a7a-a603-bdf33f6aefd7 service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:48:45.739276 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6178473-3eb6-4dc3-bf0a-9b83b6ecac47 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e16e26784601425c9883acd91f65becc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:48:45.739976 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6178473-3eb6-4dc3-bf0a-9b83b6ecac47 service swift] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:48:45.740470 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6178473-3eb6-4dc3-bf0a-9b83b6ecac47 service swift] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:48:45.740749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6178473-3eb6-4dc3-bf0a-9b83b6ecac47 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:48:45.750458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6178473-3eb6-4dc3-bf0a-9b83b6ecac47 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:48:45.752135 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6178473-3eb6-4dc3-bf0a-9b83b6ecac47 service swift] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:48:45.761122 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1388/2753] 10.222.0.121 () {60 vars in 1389 bytes} [Thu Aug 4 10:48:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:48:55.691714 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=9 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:48:55.757831 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=12 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:48:58.340818 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8226a-7675-466e-8b50-cccc87079631 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:48:58.351722 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8226a-7675-466e-8b50-cccc87079631 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:48:58.356387 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8d8b9-9ea1-4dc6-854b-9b6309cc0857 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:48:58.357041 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8d8b9-9ea1-4dc6-854b-9b6309cc0857 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:48:58.357368 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8d8b9-9ea1-4dc6-854b-9b6309cc0857 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:48:58.357623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8d8b9-9ea1-4dc6-854b-9b6309cc0857 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:48:58.359726 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-97a8d8b9-9ea1-4dc6-854b-9b6309cc0857 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:48:58.370987 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a8d8b9-9ea1-4dc6-854b-9b6309cc0857 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:48:58.372868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a8d8b9-9ea1-4dc6-854b-9b6309cc0857 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:48:58.382615 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1366/2754] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:48:58 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:49:03.475324 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb92dcd-7819-436a-bef7-5e5b078c13d4 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:49:03.476168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb92dcd-7819-436a-bef7-5e5b078c13d4 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:49:03.476168 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb92dcd-7819-436a-bef7-5e5b078c13d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:49:03.497215 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-ceb92dcd-7819-436a-bef7-5e5b078c13d4 None None] MFA Rules not processed for user `efe6e59228bd49bfbff25a11eb682426`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:49:03.498473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-ceb92dcd-7819-436a-bef7-5e5b078c13d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:49:03.506422 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1389/2755] 10.222.0.121 () {58 vars in 972 bytes} [Thu Aug 4 10:49:03 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:49:03.520951 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8d8b9-9ea1-4dc6-854b-9b6309cc0857 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:49:03.537375 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8d8b9-9ea1-4dc6-854b-9b6309cc0857 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:49:03.542979 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442b750-2f1d-4d5e-b18f-f62c67e6f63f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e16e26784601425c9883acd91f65becc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:49:03.543625 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442b750-2f1d-4d5e-b18f-f62c67e6f63f service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:49:03.543904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442b750-2f1d-4d5e-b18f-f62c67e6f63f service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:49:03.544226 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442b750-2f1d-4d5e-b18f-f62c67e6f63f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:49:03.557087 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442b750-2f1d-4d5e-b18f-f62c67e6f63f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:49:03.560106 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442b750-2f1d-4d5e-b18f-f62c67e6f63f service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:49:03.576189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1367/2756] 10.222.0.121 () {60 vars in 1389 bytes} [Thu Aug 4 10:49:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:49:13.494386 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:49:13.570109 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=12 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:03.011927 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb92dcd-7819-436a-bef7-5e5b078c13d4 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:03.014303 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-ceb92dcd-7819-436a-bef7-5e5b078c13d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:03.041158 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb92dcd-7819-436a-bef7-5e5b078c13d4 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:03.052291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9018ef8-736a-47ec-98c9-eb0a48cf6dd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:03.071714 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9018ef8-736a-47ec-98c9-eb0a48cf6dd7 service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:03.073749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9018ef8-736a-47ec-98c9-eb0a48cf6dd7 service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:03.074548 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9018ef8-736a-47ec-98c9-eb0a48cf6dd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:03.079374 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-b9018ef8-736a-47ec-98c9-eb0a48cf6dd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:03.096220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9018ef8-736a-47ec-98c9-eb0a48cf6dd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:03.096940 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9018ef8-736a-47ec-98c9-eb0a48cf6dd7 service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:03.130313 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1390/2757] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:50:03 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:50:07.437890 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442b750-2f1d-4d5e-b18f-f62c67e6f63f service swift] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:07.446022 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442b750-2f1d-4d5e-b18f-f62c67e6f63f service swift] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:07.450424 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103af76-60d4-4cb6-a0fb-9ba88da9a4fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:07.451086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103af76-60d4-4cb6-a0fb-9ba88da9a4fa service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:07.451375 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103af76-60d4-4cb6-a0fb-9ba88da9a4fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:07.451640 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103af76-60d4-4cb6-a0fb-9ba88da9a4fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:07.453772 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1103af76-60d4-4cb6-a0fb-9ba88da9a4fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:07.526651 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103af76-60d4-4cb6-a0fb-9ba88da9a4fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:07.528143 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103af76-60d4-4cb6-a0fb-9ba88da9a4fa service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:07.538846 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1368/2758] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:50:07 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 10:50:11.364507 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9018ef8-736a-47ec-98c9-eb0a48cf6dd7 service nova] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:11.378058 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9018ef8-736a-47ec-98c9-eb0a48cf6dd7 service nova] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:11.383636 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176cc76-9981-4233-9f15-1227ce272cb7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c7cfbe911c4378857848966f4f8d98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:11.384454 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176cc76-9981-4233-9f15-1227ce272cb7 service placement] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:11.384741 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176cc76-9981-4233-9f15-1227ce272cb7 service placement] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:11.385024 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176cc76-9981-4233-9f15-1227ce272cb7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:11.387024 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-7176cc76-9981-4233-9f15-1227ce272cb7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:11.401439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176cc76-9981-4233-9f15-1227ce272cb7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:11.403003 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176cc76-9981-4233-9f15-1227ce272cb7 service placement] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:11.417569 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1391/2759] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:50:11 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:50:11.440279 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103af76-60d4-4cb6-a0fb-9ba88da9a4fa service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:11.443003 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1103af76-60d4-4cb6-a0fb-9ba88da9a4fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:11.456452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103af76-60d4-4cb6-a0fb-9ba88da9a4fa service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:11.461488 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f75da-82da-4bbb-bbc2-1f0f39944b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:11.462435 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f75da-82da-4bbb-bbc2-1f0f39944b2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:11.462716 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f75da-82da-4bbb-bbc2-1f0f39944b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:11.462993 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f75da-82da-4bbb-bbc2-1f0f39944b2a admin admin] PATH_INFO: `/v3/users/f2c14f2350dd450b846db0c22c43688b` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:11.464584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f75da-82da-4bbb-bbc2-1f0f39944b2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2c14f2350dd450b846db0c22c43688b)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:11.466522 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f75da-82da-4bbb-bbc2-1f0f39944b2a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:11.539014 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2d6f75da-82da-4bbb-bbc2-1f0f39944b2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2c14f2350dd450b846db0c22c43688b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:11.548182 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2d6f75da-82da-4bbb-bbc2-1f0f39944b2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2c14f2350dd450b846db0c22c43688b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:11.549021 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2d6f75da-82da-4bbb-bbc2-1f0f39944b2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2c14f2350dd450b846db0c22c43688b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:11.565814 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2d6f75da-82da-4bbb-bbc2-1f0f39944b2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2c14f2350dd450b846db0c22c43688b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:11.566998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2d6f75da-82da-4bbb-bbc2-1f0f39944b2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2c14f2350dd450b846db0c22c43688b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:11.584603 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1369/2760] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:50:11 2022] DELETE /identity/v3/users/f2c14f2350dd450b846db0c22c43688b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:11.789957 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176cc76-9981-4233-9f15-1227ce272cb7 service placement] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:11.805656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176cc76-9981-4233-9f15-1227ce272cb7 service placement] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:11.880508 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ab1fd-7795-4963-b6ee-527fa0ba37e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:11.882804 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ab1fd-7795-4963-b6ee-527fa0ba37e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:11.883081 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ab1fd-7795-4963-b6ee-527fa0ba37e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:11.883387 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ab1fd-7795-4963-b6ee-527fa0ba37e3 admin admin] PATH_INFO: `/v3/projects/18d95fd38a664ce1b69afbb11189020a` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:11.884971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ab1fd-7795-4963-b6ee-527fa0ba37e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18d95fd38a664ce1b69afbb11189020a)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:11.886973 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ab1fd-7795-4963-b6ee-527fa0ba37e3 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:11.948755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f55ab1fd-7795-4963-b6ee-527fa0ba37e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18d95fd38a664ce1b69afbb11189020a', 'request_id': 'req-f55ab1fd-7795-4963-b6ee-527fa0ba37e3'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:11.960907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f55ab1fd-7795-4963-b6ee-527fa0ba37e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18d95fd38a664ce1b69afbb11189020a', 'request_id': 'req-f55ab1fd-7795-4963-b6ee-527fa0ba37e3'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:11.970027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f55ab1fd-7795-4963-b6ee-527fa0ba37e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18d95fd38a664ce1b69afbb11189020a', 'request_id': 'req-f55ab1fd-7795-4963-b6ee-527fa0ba37e3'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:11.983256 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f55ab1fd-7795-4963-b6ee-527fa0ba37e3 admin admin] The token cache is being invalidate because project 18d95fd38a664ce1b69afbb11189020a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:50:11.983544 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f55ab1fd-7795-4963-b6ee-527fa0ba37e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:11.984271 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f55ab1fd-7795-4963-b6ee-527fa0ba37e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:11.985928 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1392/2761] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:50:11 2022] DELETE /identity/v3/projects/18d95fd38a664ce1b69afbb11189020a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:11.991758 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f8544-c628-4576-a6f4-d80258e30b1b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:11.992176 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f8544-c628-4576-a6f4-d80258e30b1b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:11.992463 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f8544-c628-4576-a6f4-d80258e30b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:12.014888 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-214f8544-c628-4576-a6f4-d80258e30b1b None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:12.016205 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-214f8544-c628-4576-a6f4-d80258e30b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:12.089705 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1370/2762] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:50:11 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:50:12.099490 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874a2cd-d49f-42e0-92f5-24f29f408551 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:12.099791 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874a2cd-d49f-42e0-92f5-24f29f408551 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:12.100077 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874a2cd-d49f-42e0-92f5-24f29f408551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:12.122127 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-3874a2cd-d49f-42e0-92f5-24f29f408551 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:12.123410 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-3874a2cd-d49f-42e0-92f5-24f29f408551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:12.132138 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1393/2763] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:50:12 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:50:12.137048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f8544-c628-4576-a6f4-d80258e30b1b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:12.147799 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f8544-c628-4576-a6f4-d80258e30b1b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:12.154748 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079eb08-5cda-473d-a230-53ec0dd299b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:12.155420 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079eb08-5cda-473d-a230-53ec0dd299b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:12.155705 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079eb08-5cda-473d-a230-53ec0dd299b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:12.156010 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079eb08-5cda-473d-a230-53ec0dd299b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:12.156743 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9079eb08-5cda-473d-a230-53ec0dd299b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:50:12.157045 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9079eb08-5cda-473d-a230-53ec0dd299b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:12.158729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9079eb08-5cda-473d-a230-53ec0dd299b4 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:12.160595 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:12.175972 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1371/2764] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:50:12 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:50:12.180909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874a2cd-d49f-42e0-92f5-24f29f408551 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:12.191780 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874a2cd-d49f-42e0-92f5-24f29f408551 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:12.198306 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fcce2-ce5f-4f09-8688-3bccfdcfcb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:12.199047 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fcce2-ce5f-4f09-8688-3bccfdcfcb41 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:12.199261 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fcce2-ce5f-4f09-8688-3bccfdcfcb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:12.199622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fcce2-ce5f-4f09-8688-3bccfdcfcb41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:12.200506 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796fcce2-ce5f-4f09-8688-3bccfdcfcb41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:12.203601 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796fcce2-ce5f-4f09-8688-3bccfdcfcb41 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:12.239288 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1394/2765] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:50:12 2022] POST /identity/v3/projects => generated 402 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:50:12.244226 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079eb08-5cda-473d-a230-53ec0dd299b4 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:12.256173 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079eb08-5cda-473d-a230-53ec0dd299b4 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:12.329372 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce8879-2df5-4dbb-b156-95a6b2d97171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:12.330263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce8879-2df5-4dbb-b156-95a6b2d97171 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:12.330732 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce8879-2df5-4dbb-b156-95a6b2d97171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:12.331216 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce8879-2df5-4dbb-b156-95a6b2d97171 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:12.332408 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffce8879-2df5-4dbb-b156-95a6b2d97171 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:12.335537 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffce8879-2df5-4dbb-b156-95a6b2d97171 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:12.363055 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1372/2766] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:50:12 2022] POST /identity/v3/users => generated 302 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:50:12.368072 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fcce2-ce5f-4f09-8688-3bccfdcfcb41 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:12.380512 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fcce2-ce5f-4f09-8688-3bccfdcfcb41 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:12.385692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c228114-9770-4080-a96d-fd0ad5031eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:12.386341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c228114-9770-4080-a96d-fd0ad5031eba admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:12.386621 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c228114-9770-4080-a96d-fd0ad5031eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:12.387099 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c228114-9770-4080-a96d-fd0ad5031eba admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:12.387859 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c228114-9770-4080-a96d-fd0ad5031eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:12.389606 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c228114-9770-4080-a96d-fd0ad5031eba admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:12.401313 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1395/2767] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:50:12 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:50:12.407649 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce8879-2df5-4dbb-b156-95a6b2d97171 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:12.419589 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce8879-2df5-4dbb-b156-95a6b2d97171 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:12.425678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4d78e-a91f-458b-b6d9-008cd76cda5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:12.426317 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4d78e-a91f-458b-b6d9-008cd76cda5d admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:12.426600 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4d78e-a91f-458b-b6d9-008cd76cda5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:12.426877 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4d78e-a91f-458b-b6d9-008cd76cda5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:12.427627 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4d78e-a91f-458b-b6d9-008cd76cda5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:12.429356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4d78e-a91f-458b-b6d9-008cd76cda5d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:12.443462 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1373/2768] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:50:12 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:50:12.448286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c228114-9770-4080-a96d-fd0ad5031eba admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:12.459533 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c228114-9770-4080-a96d-fd0ad5031eba admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:12.465680 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62a4f8-beb6-49fa-bebe-f3d386966b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:12.466502 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62a4f8-beb6-49fa-bebe-f3d386966b37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:12.466807 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62a4f8-beb6-49fa-bebe-f3d386966b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:12.467085 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62a4f8-beb6-49fa-bebe-f3d386966b37 admin admin] PATH_INFO: `/v3/projects/6d6d83b2c9db4b259d3be06ed5517846/users/a42f2930bbcf440f81236aa192a8f2d3/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:12.484751 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62a4f8-beb6-49fa-bebe-f3d386966b37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d6d83b2c9db4b259d3be06ed5517846, user_id=a42f2930bbcf440f81236aa192a8f2d3, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:12.487487 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62a4f8-beb6-49fa-bebe-f3d386966b37 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:12.498847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:12.504715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1396/2769] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:50:12 2022] PUT /identity/v3/projects/6d6d83b2c9db4b259d3be06ed5517846/users/a42f2930bbcf440f81236aa192a8f2d3/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:12.510660 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d26df19-aefa-445b-a945-587f0de3bea3 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:12.511062 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d26df19-aefa-445b-a945-587f0de3bea3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:12.511347 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d26df19-aefa-445b-a945-587f0de3bea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:12.534168 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4d26df19-aefa-445b-a945-587f0de3bea3 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:12.535498 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4d26df19-aefa-445b-a945-587f0de3bea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:12.602404 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1374/2770] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:50:12 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:50:12.613933 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62a4f8-beb6-49fa-bebe-f3d386966b37 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:12.616039 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-1d62a4f8-beb6-49fa-bebe-f3d386966b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:12.687143 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62a4f8-beb6-49fa-bebe-f3d386966b37 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:12.691565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77c4d0-a6ee-4002-8b6d-17d47c2bbd59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:12.692275 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77c4d0-a6ee-4002-8b6d-17d47c2bbd59 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:12.692544 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77c4d0-a6ee-4002-8b6d-17d47c2bbd59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:12.692823 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77c4d0-a6ee-4002-8b6d-17d47c2bbd59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:12.703754 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb77c4d0-a6ee-4002-8b6d-17d47c2bbd59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:12.705287 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb77c4d0-a6ee-4002-8b6d-17d47c2bbd59 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:12.716869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1397/2771] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:50:12 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:50:13.526632 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a92fb-ca1c-418a-a6e0-b9d0aad72934 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:13.527334 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a92fb-ca1c-418a-a6e0-b9d0aad72934 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:13.527527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a92fb-ca1c-418a-a6e0-b9d0aad72934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:13.588467 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8a1a92fb-ca1c-418a-a6e0-b9d0aad72934 None None] MFA Rules not processed for user `a42f2930bbcf440f81236aa192a8f2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:13.591048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8a1a92fb-ca1c-418a-a6e0-b9d0aad72934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:13.685604 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1375/2772] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:50:13 2022] POST /identity/v3/auth/tokens => generated 3288 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:50:13.690971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77c4d0-a6ee-4002-8b6d-17d47c2bbd59 service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:13.702078 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77c4d0-a6ee-4002-8b6d-17d47c2bbd59 service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:13.710885 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa6b552-24bc-4826-9d1c-7ecbd958a7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:13.711547 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6b552-24bc-4826-9d1c-7ecbd958a7e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:13.711830 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6b552-24bc-4826-9d1c-7ecbd958a7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:13.712146 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6b552-24bc-4826-9d1c-7ecbd958a7e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:13.712858 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa6b552-24bc-4826-9d1c-7ecbd958a7e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:13.715191 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa6b552-24bc-4826-9d1c-7ecbd958a7e0 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:13.752395 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1398/2773] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:50:13 2022] POST /identity/v3/projects => generated 402 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:50:13.757215 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a92fb-ca1c-418a-a6e0-b9d0aad72934 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:13.763449 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:13.774061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a92fb-ca1c-418a-a6e0-b9d0aad72934 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:13.847652 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88fd6a3-213f-4c78-896a-2e138dfacfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:13.848357 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fd6a3-213f-4c78-896a-2e138dfacfdc admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:13.848641 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fd6a3-213f-4c78-896a-2e138dfacfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:13.848951 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fd6a3-213f-4c78-896a-2e138dfacfdc admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:13.849628 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88fd6a3-213f-4c78-896a-2e138dfacfdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:13.851688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88fd6a3-213f-4c78-896a-2e138dfacfdc admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:13.876150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1376/2774] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:50:13 2022] POST /identity/v3/users => generated 308 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:50:13.880651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa6b552-24bc-4826-9d1c-7ecbd958a7e0 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:13.894330 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa6b552-24bc-4826-9d1c-7ecbd958a7e0 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:13.900388 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48fbf4-a18d-4663-b335-52a6e9933296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:13.901194 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48fbf4-a18d-4663-b335-52a6e9933296 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:13.901480 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48fbf4-a18d-4663-b335-52a6e9933296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:13.901763 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48fbf4-a18d-4663-b335-52a6e9933296 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:13.902485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48fbf4-a18d-4663-b335-52a6e9933296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:13.904182 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48fbf4-a18d-4663-b335-52a6e9933296 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:14.053294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1399/2775] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:50:13 2022] GET /identity/v3/roles => generated 1427 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:50:14.058460 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88fd6a3-213f-4c78-896a-2e138dfacfdc admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:14.070502 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88fd6a3-213f-4c78-896a-2e138dfacfdc admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:14.083044 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe0762-5c9e-46d7-b6af-9bac57cb892d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:14.083755 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe0762-5c9e-46d7-b6af-9bac57cb892d admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:14.084087 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe0762-5c9e-46d7-b6af-9bac57cb892d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:14.084371 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe0762-5c9e-46d7-b6af-9bac57cb892d admin admin] PATH_INFO: `/v3/projects/84ede77203b74070a1658dc0f2cff82e/users/472e429698724aaba3b36515bcda69e1/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:14.105621 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe0762-5c9e-46d7-b6af-9bac57cb892d admin admin] RBAC: Authorizing `identity:create_grant(project_id=84ede77203b74070a1658dc0f2cff82e, user_id=472e429698724aaba3b36515bcda69e1, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:14.109419 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe0762-5c9e-46d7-b6af-9bac57cb892d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:14.126904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1377/2776] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:50:14 2022] PUT /identity/v3/projects/84ede77203b74070a1658dc0f2cff82e/users/472e429698724aaba3b36515bcda69e1/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:14.131865 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48fbf4-a18d-4663-b335-52a6e9933296 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:14.146446 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48fbf4-a18d-4663-b335-52a6e9933296 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:14.207719 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946d713-ba80-4bfe-8498-ec250f370046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:14.208396 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946d713-ba80-4bfe-8498-ec250f370046 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:14.208676 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946d713-ba80-4bfe-8498-ec250f370046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:14.208955 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946d713-ba80-4bfe-8498-ec250f370046 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:14.209682 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946d713-ba80-4bfe-8498-ec250f370046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:14.211349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946d713-ba80-4bfe-8498-ec250f370046 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:14.223529 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1400/2777] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:50:14 2022] GET /identity/v3/roles => generated 1427 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:50:14.229894 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe0762-5c9e-46d7-b6af-9bac57cb892d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:14.243354 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe0762-5c9e-46d7-b6af-9bac57cb892d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:14.248955 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81cd03-ad7d-4c16-ad74-6f05c62b0fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:14.249657 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81cd03-ad7d-4c16-ad74-6f05c62b0fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:14.249942 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81cd03-ad7d-4c16-ad74-6f05c62b0fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:14.250230 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81cd03-ad7d-4c16-ad74-6f05c62b0fc2 admin admin] PATH_INFO: `/v3/projects/84ede77203b74070a1658dc0f2cff82e/users/472e429698724aaba3b36515bcda69e1/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:14.252376 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f81cd03-ad7d-4c16-ad74-6f05c62b0fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84ede77203b74070a1658dc0f2cff82e, user_id=472e429698724aaba3b36515bcda69e1, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:14.255133 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f81cd03-ad7d-4c16-ad74-6f05c62b0fc2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:14.267145 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1378/2778] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:50:14 2022] PUT /identity/v3/projects/84ede77203b74070a1658dc0f2cff82e/users/472e429698724aaba3b36515bcda69e1/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:14.999109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502dcde1-b0bf-4082-ba0a-ce0d2626c0ee None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:14.999715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502dcde1-b0bf-4082-ba0a-ce0d2626c0ee None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:14.999715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502dcde1-b0bf-4082-ba0a-ce0d2626c0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:15.031025 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-502dcde1-b0bf-4082-ba0a-ce0d2626c0ee None None] MFA Rules not processed for user `472e429698724aaba3b36515bcda69e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:15.032357 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-502dcde1-b0bf-4082-ba0a-ce0d2626c0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:15.100127 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:15.131549 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1401/2779] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:50:14 2022] POST /identity/v3/auth/tokens => generated 3355 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:50:15.662994 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81cd03-ad7d-4c16-ad74-6f05c62b0fc2 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:15.665114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7f81cd03-ad7d-4c16-ad74-6f05c62b0fc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:15.757061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81cd03-ad7d-4c16-ad74-6f05c62b0fc2 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:15.764937 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975308d-dbce-4e75-8f97-559298154943 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c7cfbe911c4378857848966f4f8d98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:15.765622 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975308d-dbce-4e75-8f97-559298154943 service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:15.765907 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975308d-dbce-4e75-8f97-559298154943 service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:15.766181 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975308d-dbce-4e75-8f97-559298154943 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:15.768169 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e975308d-dbce-4e75-8f97-559298154943 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:15.790779 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975308d-dbce-4e75-8f97-559298154943 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:15.794158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975308d-dbce-4e75-8f97-559298154943 service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:15.810794 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1379/2780] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:50:15 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:50:16.364125 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502dcde1-b0bf-4082-ba0a-ce0d2626c0ee None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:16.434602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502dcde1-b0bf-4082-ba0a-ce0d2626c0ee None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:16.439907 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b1bd06-cf7b-4b30-81cb-e58242705622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:16.440705 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b1bd06-cf7b-4b30-81cb-e58242705622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:16.440904 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b1bd06-cf7b-4b30-81cb-e58242705622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:16.441237 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b1bd06-cf7b-4b30-81cb-e58242705622 admin admin] PATH_INFO: `/v3/users/a42f2930bbcf440f81236aa192a8f2d3` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:16.442998 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b1bd06-cf7b-4b30-81cb-e58242705622 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a42f2930bbcf440f81236aa192a8f2d3)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:16.445206 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b1bd06-cf7b-4b30-81cb-e58242705622 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:16.505510 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-29b1bd06-cf7b-4b30-81cb-e58242705622 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a42f2930bbcf440f81236aa192a8f2d3'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:16.506249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-29b1bd06-cf7b-4b30-81cb-e58242705622 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a42f2930bbcf440f81236aa192a8f2d3'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:16.521094 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-29b1bd06-cf7b-4b30-81cb-e58242705622 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a42f2930bbcf440f81236aa192a8f2d3'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:16.537851 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-29b1bd06-cf7b-4b30-81cb-e58242705622 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a42f2930bbcf440f81236aa192a8f2d3'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:16.547603 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-29b1bd06-cf7b-4b30-81cb-e58242705622 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a42f2930bbcf440f81236aa192a8f2d3'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:16.555284 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1402/2781] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:50:16 2022] DELETE /identity/v3/users/a42f2930bbcf440f81236aa192a8f2d3 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:16.775514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975308d-dbce-4e75-8f97-559298154943 service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:16.790082 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975308d-dbce-4e75-8f97-559298154943 service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:16.849062 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:16.871251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1a2e6-cc95-4bed-8192-6cfe6aa1ed20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:16.871915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1a2e6-cc95-4bed-8192-6cfe6aa1ed20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:16.872233 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1a2e6-cc95-4bed-8192-6cfe6aa1ed20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:16.872583 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1a2e6-cc95-4bed-8192-6cfe6aa1ed20 admin admin] PATH_INFO: `/v3/projects/6d6d83b2c9db4b259d3be06ed5517846` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:16.873856 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1a2e6-cc95-4bed-8192-6cfe6aa1ed20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d6d83b2c9db4b259d3be06ed5517846)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:16.875805 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1a2e6-cc95-4bed-8192-6cfe6aa1ed20 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:16.938257 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-15f1a2e6-cc95-4bed-8192-6cfe6aa1ed20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d6d83b2c9db4b259d3be06ed5517846', 'request_id': 'req-15f1a2e6-cc95-4bed-8192-6cfe6aa1ed20'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:16.945448 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-15f1a2e6-cc95-4bed-8192-6cfe6aa1ed20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d6d83b2c9db4b259d3be06ed5517846', 'request_id': 'req-15f1a2e6-cc95-4bed-8192-6cfe6aa1ed20'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:16.952119 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-15f1a2e6-cc95-4bed-8192-6cfe6aa1ed20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d6d83b2c9db4b259d3be06ed5517846', 'request_id': 'req-15f1a2e6-cc95-4bed-8192-6cfe6aa1ed20'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:16.969121 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-15f1a2e6-cc95-4bed-8192-6cfe6aa1ed20 admin admin] The token cache is being invalidate because project 6d6d83b2c9db4b259d3be06ed5517846 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:50:16.969505 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-15f1a2e6-cc95-4bed-8192-6cfe6aa1ed20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:16.970315 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-15f1a2e6-cc95-4bed-8192-6cfe6aa1ed20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:16.972165 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1380/2782] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:50:16 2022] DELETE /identity/v3/projects/6d6d83b2c9db4b259d3be06ed5517846 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:16.977657 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b1bd06-cf7b-4b30-81cb-e58242705622 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:16.980099 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-29b1bd06-cf7b-4b30-81cb-e58242705622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:16.998923 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b1bd06-cf7b-4b30-81cb-e58242705622 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:17.069183 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a0014-b2b1-4e71-aa2d-2c2ac1b32c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:17.069867 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a0014-b2b1-4e71-aa2d-2c2ac1b32c70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:17.070149 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a0014-b2b1-4e71-aa2d-2c2ac1b32c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:17.070425 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a0014-b2b1-4e71-aa2d-2c2ac1b32c70 admin admin] PATH_INFO: `/v3/users/472e429698724aaba3b36515bcda69e1` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:17.071649 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a0014-b2b1-4e71-aa2d-2c2ac1b32c70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=472e429698724aaba3b36515bcda69e1)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:17.073698 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a0014-b2b1-4e71-aa2d-2c2ac1b32c70 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:17.087881 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:17.132630 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-744a0014-b2b1-4e71-aa2d-2c2ac1b32c70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '472e429698724aaba3b36515bcda69e1'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:17.133480 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-744a0014-b2b1-4e71-aa2d-2c2ac1b32c70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '472e429698724aaba3b36515bcda69e1'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:17.147916 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-744a0014-b2b1-4e71-aa2d-2c2ac1b32c70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '472e429698724aaba3b36515bcda69e1'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:17.155903 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-744a0014-b2b1-4e71-aa2d-2c2ac1b32c70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '472e429698724aaba3b36515bcda69e1'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:17.165450 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-744a0014-b2b1-4e71-aa2d-2c2ac1b32c70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '472e429698724aaba3b36515bcda69e1'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:17.172503 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1403/2783] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:50:16 2022] DELETE /identity/v3/users/472e429698724aaba3b36515bcda69e1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:17.395225 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1a2e6-cc95-4bed-8192-6cfe6aa1ed20 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:17.397265 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-15f1a2e6-cc95-4bed-8192-6cfe6aa1ed20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:17.418164 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1a2e6-cc95-4bed-8192-6cfe6aa1ed20 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:17.488032 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed67b3-a319-43c7-94b7-24d8b9b826b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:17.488785 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed67b3-a319-43c7-94b7-24d8b9b826b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:17.488991 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed67b3-a319-43c7-94b7-24d8b9b826b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:17.489252 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed67b3-a319-43c7-94b7-24d8b9b826b2 admin admin] PATH_INFO: `/v3/projects/84ede77203b74070a1658dc0f2cff82e` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:17.490675 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed67b3-a319-43c7-94b7-24d8b9b826b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84ede77203b74070a1658dc0f2cff82e)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:17.492667 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed67b3-a319-43c7-94b7-24d8b9b826b2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:17.547290 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f9ed67b3-a319-43c7-94b7-24d8b9b826b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84ede77203b74070a1658dc0f2cff82e', 'request_id': 'req-f9ed67b3-a319-43c7-94b7-24d8b9b826b2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:17.556507 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f9ed67b3-a319-43c7-94b7-24d8b9b826b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84ede77203b74070a1658dc0f2cff82e', 'request_id': 'req-f9ed67b3-a319-43c7-94b7-24d8b9b826b2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:17.564203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f9ed67b3-a319-43c7-94b7-24d8b9b826b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84ede77203b74070a1658dc0f2cff82e', 'request_id': 'req-f9ed67b3-a319-43c7-94b7-24d8b9b826b2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:17.590307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f9ed67b3-a319-43c7-94b7-24d8b9b826b2 admin admin] The token cache is being invalidate because project 84ede77203b74070a1658dc0f2cff82e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:50:17.591148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f9ed67b3-a319-43c7-94b7-24d8b9b826b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:17.592147 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f9ed67b3-a319-43c7-94b7-24d8b9b826b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:17.593983 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1381/2784] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:50:17 2022] DELETE /identity/v3/projects/84ede77203b74070a1658dc0f2cff82e => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:17.599606 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb4241-7c92-4e34-8400-f980600e20d0 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:17.600177 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb4241-7c92-4e34-8400-f980600e20d0 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:17.600491 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb4241-7c92-4e34-8400-f980600e20d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:17.624869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-5afb4241-7c92-4e34-8400-f980600e20d0 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:17.626146 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-5afb4241-7c92-4e34-8400-f980600e20d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:17.700641 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1404/2785] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:50:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:50:17.712177 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014123fd-b646-43dd-89a8-98fee35c0e79 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:17.712594 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014123fd-b646-43dd-89a8-98fee35c0e79 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:17.713124 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014123fd-b646-43dd-89a8-98fee35c0e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:17.723339 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:17.736947 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-014123fd-b646-43dd-89a8-98fee35c0e79 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:17.739468 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-014123fd-b646-43dd-89a8-98fee35c0e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:17.749125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1382/2786] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:50:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:50:17.753606 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb4241-7c92-4e34-8400-f980600e20d0 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:17.764249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb4241-7c92-4e34-8400-f980600e20d0 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:17.770041 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92c979-93c6-4576-9eff-ebbdfaaf7466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:17.770767 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92c979-93c6-4576-9eff-ebbdfaaf7466 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:17.771141 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92c979-93c6-4576-9eff-ebbdfaaf7466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:17.771474 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92c979-93c6-4576-9eff-ebbdfaaf7466 admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:17.772308 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92c979-93c6-4576-9eff-ebbdfaaf7466 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:50:17.772694 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92c979-93c6-4576-9eff-ebbdfaaf7466 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:17.774380 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92c979-93c6-4576-9eff-ebbdfaaf7466 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:17.788242 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1405/2787] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:50:17 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:50:17.794104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014123fd-b646-43dd-89a8-98fee35c0e79 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:17.804818 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014123fd-b646-43dd-89a8-98fee35c0e79 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:17.810497 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02978b2-259e-4ffe-9ccb-214c5e820df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:17.811191 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02978b2-259e-4ffe-9ccb-214c5e820df3 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:17.811470 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02978b2-259e-4ffe-9ccb-214c5e820df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:17.811769 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02978b2-259e-4ffe-9ccb-214c5e820df3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:17.812503 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02978b2-259e-4ffe-9ccb-214c5e820df3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:17.814450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02978b2-259e-4ffe-9ccb-214c5e820df3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:17.847249 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1383/2788] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:50:17 2022] POST /identity/v3/projects => generated 374 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:50:17.852134 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92c979-93c6-4576-9eff-ebbdfaaf7466 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:17.868365 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92c979-93c6-4576-9eff-ebbdfaaf7466 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:17.937559 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76333c81-8ee3-4b78-8fbf-8627d3222023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:17.938239 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76333c81-8ee3-4b78-8fbf-8627d3222023 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:17.938578 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76333c81-8ee3-4b78-8fbf-8627d3222023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:17.939034 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76333c81-8ee3-4b78-8fbf-8627d3222023 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:17.939839 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76333c81-8ee3-4b78-8fbf-8627d3222023 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:17.942053 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76333c81-8ee3-4b78-8fbf-8627d3222023 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:17.967030 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1406/2789] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:50:17 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:50:17.971782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02978b2-259e-4ffe-9ccb-214c5e820df3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:17.985237 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02978b2-259e-4ffe-9ccb-214c5e820df3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:17.991363 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33dabe6-fa2e-43b2-aab7-40a26eaa0b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:17.992127 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33dabe6-fa2e-43b2-aab7-40a26eaa0b4a admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:17.992415 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33dabe6-fa2e-43b2-aab7-40a26eaa0b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:17.992696 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33dabe6-fa2e-43b2-aab7-40a26eaa0b4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:17.993499 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33dabe6-fa2e-43b2-aab7-40a26eaa0b4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:17.995101 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33dabe6-fa2e-43b2-aab7-40a26eaa0b4a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:18.007343 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1384/2790] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:50:17 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:50:18.012171 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76333c81-8ee3-4b78-8fbf-8627d3222023 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:18.023646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76333c81-8ee3-4b78-8fbf-8627d3222023 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:18.029294 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9681186-ded5-4802-b978-2aa2d30c0c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:18.029990 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9681186-ded5-4802-b978-2aa2d30c0c15 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:18.030317 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9681186-ded5-4802-b978-2aa2d30c0c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:18.030630 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9681186-ded5-4802-b978-2aa2d30c0c15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:18.031389 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9681186-ded5-4802-b978-2aa2d30c0c15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:18.033383 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9681186-ded5-4802-b978-2aa2d30c0c15 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:18.036534 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:18.054115 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1407/2791] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:50:18 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:50:18.059237 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33dabe6-fa2e-43b2-aab7-40a26eaa0b4a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:18.070274 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33dabe6-fa2e-43b2-aab7-40a26eaa0b4a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:18.075313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2fb8d-651b-4d29-a7ab-696bdc171199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:18.076020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2fb8d-651b-4d29-a7ab-696bdc171199 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:18.076304 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2fb8d-651b-4d29-a7ab-696bdc171199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:18.076588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2fb8d-651b-4d29-a7ab-696bdc171199 admin admin] PATH_INFO: `/v3/projects/454a36e8b9de41af97b1d8e7dca8f2bd/users/23d2bacbf5d643dda9d6e316ac3355d4/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:18.096855 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2fb8d-651b-4d29-a7ab-696bdc171199 admin admin] RBAC: Authorizing `identity:create_grant(project_id=454a36e8b9de41af97b1d8e7dca8f2bd, user_id=23d2bacbf5d643dda9d6e316ac3355d4, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:18.099553 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2fb8d-651b-4d29-a7ab-696bdc171199 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:18.117414 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1385/2792] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:50:18 2022] PUT /identity/v3/projects/454a36e8b9de41af97b1d8e7dca8f2bd/users/23d2bacbf5d643dda9d6e316ac3355d4/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:18.129592 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93b7a6-cee9-4ac0-a6bc-87c8af4977e9 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:18.129956 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93b7a6-cee9-4ac0-a6bc-87c8af4977e9 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:18.130279 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93b7a6-cee9-4ac0-a6bc-87c8af4977e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:18.166949 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-6f93b7a6-cee9-4ac0-a6bc-87c8af4977e9 None None] MFA Rules not processed for user `23d2bacbf5d643dda9d6e316ac3355d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:18.168610 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6f93b7a6-cee9-4ac0-a6bc-87c8af4977e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:18.241677 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1408/2793] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:50:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:50:18.254146 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2fb8d-651b-4d29-a7ab-696bdc171199 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:18.256422 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0ed2fb8d-651b-4d29-a7ab-696bdc171199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:18.320672 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2fb8d-651b-4d29-a7ab-696bdc171199 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:18.324542 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ada5f-0aea-4987-b635-abb5be04e927 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:18.325182 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ada5f-0aea-4987-b635-abb5be04e927 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:18.325465 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ada5f-0aea-4987-b635-abb5be04e927 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:18.325760 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ada5f-0aea-4987-b635-abb5be04e927 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:18.335975 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ada5f-0aea-4987-b635-abb5be04e927 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:18.337434 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ada5f-0aea-4987-b635-abb5be04e927 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:18.355494 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1386/2794] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:50:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:50:21.578216 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93b7a6-cee9-4ac0-a6bc-87c8af4977e9 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:21.656819 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93b7a6-cee9-4ac0-a6bc-87c8af4977e9 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:21.663268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4ab90-bea1-4777-8ea5-5ab70705073c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:21.663981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4ab90-bea1-4777-8ea5-5ab70705073c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:21.664292 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4ab90-bea1-4777-8ea5-5ab70705073c admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:21.664566 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4ab90-bea1-4777-8ea5-5ab70705073c admin admin] PATH_INFO: `/v3/users/23d2bacbf5d643dda9d6e316ac3355d4` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:21.665901 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4ab90-bea1-4777-8ea5-5ab70705073c admin admin] RBAC: Authorizing `identity:delete_user(user_id=23d2bacbf5d643dda9d6e316ac3355d4)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:21.667963 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4ab90-bea1-4777-8ea5-5ab70705073c admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:21.735158 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-baf4ab90-bea1-4777-8ea5-5ab70705073c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23d2bacbf5d643dda9d6e316ac3355d4'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:21.735855 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-baf4ab90-bea1-4777-8ea5-5ab70705073c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23d2bacbf5d643dda9d6e316ac3355d4'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:21.749268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-baf4ab90-bea1-4777-8ea5-5ab70705073c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23d2bacbf5d643dda9d6e316ac3355d4'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:21.758420 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-baf4ab90-bea1-4777-8ea5-5ab70705073c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23d2bacbf5d643dda9d6e316ac3355d4'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:21.779927 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-baf4ab90-bea1-4777-8ea5-5ab70705073c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23d2bacbf5d643dda9d6e316ac3355d4'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:21.789186 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1409/2795] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:50:21 2022] DELETE /identity/v3/users/23d2bacbf5d643dda9d6e316ac3355d4 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:21.795013 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d24939-c461-49f3-b6be-92074610b62c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:21.795898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d24939-c461-49f3-b6be-92074610b62c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:21.795898 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d24939-c461-49f3-b6be-92074610b62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:21.821656 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-02d24939-c461-49f3-b6be-92074610b62c None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:21.823021 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-02d24939-c461-49f3-b6be-92074610b62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:21.900988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1387/2796] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:50:21 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:50:21.914019 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4ab90-bea1-4777-8ea5-5ab70705073c admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:21.915825 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-baf4ab90-bea1-4777-8ea5-5ab70705073c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:21.924017 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:21.993538 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4ab90-bea1-4777-8ea5-5ab70705073c admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:21.998037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebd5a2-f8fc-409e-a1ff-d2be1ad849b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:21.998709 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebd5a2-f8fc-409e-a1ff-d2be1ad849b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:21.999004 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebd5a2-f8fc-409e-a1ff-d2be1ad849b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:21.999279 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebd5a2-f8fc-409e-a1ff-d2be1ad849b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:22.010341 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ebd5a2-f8fc-409e-a1ff-d2be1ad849b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:22.011927 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ebd5a2-f8fc-409e-a1ff-d2be1ad849b1 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:22.030844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1410/2797] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:50:21 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:50:22.405162 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d24939-c461-49f3-b6be-92074610b62c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:22.407298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-02d24939-c461-49f3-b6be-92074610b62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:22.429491 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d24939-c461-49f3-b6be-92074610b62c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:22.440639 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35094822-f9fa-41e9-a49e-aab43dcaf0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:22.441253 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35094822-f9fa-41e9-a49e-aab43dcaf0cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:22.441637 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35094822-f9fa-41e9-a49e-aab43dcaf0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:22.441796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35094822-f9fa-41e9-a49e-aab43dcaf0cb admin admin] PATH_INFO: `/v3/projects/454a36e8b9de41af97b1d8e7dca8f2bd` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:22.443609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35094822-f9fa-41e9-a49e-aab43dcaf0cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=454a36e8b9de41af97b1d8e7dca8f2bd)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:22.445616 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35094822-f9fa-41e9-a49e-aab43dcaf0cb admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:22.513789 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:22.525455 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-35094822-f9fa-41e9-a49e-aab43dcaf0cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '454a36e8b9de41af97b1d8e7dca8f2bd', 'request_id': 'req-35094822-f9fa-41e9-a49e-aab43dcaf0cb'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:22.535976 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-35094822-f9fa-41e9-a49e-aab43dcaf0cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '454a36e8b9de41af97b1d8e7dca8f2bd', 'request_id': 'req-35094822-f9fa-41e9-a49e-aab43dcaf0cb'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:22.548584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-35094822-f9fa-41e9-a49e-aab43dcaf0cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '454a36e8b9de41af97b1d8e7dca8f2bd', 'request_id': 'req-35094822-f9fa-41e9-a49e-aab43dcaf0cb'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:22.568132 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-35094822-f9fa-41e9-a49e-aab43dcaf0cb admin admin] The token cache is being invalidate because project 454a36e8b9de41af97b1d8e7dca8f2bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:50:22.568588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-35094822-f9fa-41e9-a49e-aab43dcaf0cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:22.569570 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-35094822-f9fa-41e9-a49e-aab43dcaf0cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:22.571598 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1388/2798] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:50:22 2022] DELETE /identity/v3/projects/454a36e8b9de41af97b1d8e7dca8f2bd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:22.580372 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe5506-abc3-4372-ae3e-bb217db5ce07 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:22.581052 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe5506-abc3-4372-ae3e-bb217db5ce07 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:22.581182 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe5506-abc3-4372-ae3e-bb217db5ce07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:22.608537 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-f1fe5506-abc3-4372-ae3e-bb217db5ce07 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:22.609825 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-f1fe5506-abc3-4372-ae3e-bb217db5ce07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:22.690087 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1411/2799] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:50:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:50:22.704131 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80a748-eb43-48fd-b960-a8c156ff9195 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:22.704478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80a748-eb43-48fd-b960-a8c156ff9195 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:22.704808 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80a748-eb43-48fd-b960-a8c156ff9195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:22.732389 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7b80a748-eb43-48fd-b960-a8c156ff9195 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:22.733660 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7b80a748-eb43-48fd-b960-a8c156ff9195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:22.742826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1389/2800] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:50:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:50:22.747667 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe5506-abc3-4372-ae3e-bb217db5ce07 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:22.758821 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe5506-abc3-4372-ae3e-bb217db5ce07 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:22.764158 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249cf53a-29ef-465f-8d99-153e2c3dd8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:22.764863 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cf53a-29ef-465f-8d99-153e2c3dd8c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:22.765192 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cf53a-29ef-465f-8d99-153e2c3dd8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:22.765515 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cf53a-29ef-465f-8d99-153e2c3dd8c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:22.766285 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249cf53a-29ef-465f-8d99-153e2c3dd8c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:50:22.766648 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249cf53a-29ef-465f-8d99-153e2c3dd8c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:22.768397 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249cf53a-29ef-465f-8d99-153e2c3dd8c5 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:22.782026 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1412/2801] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:50:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:50:22.787986 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80a748-eb43-48fd-b960-a8c156ff9195 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:22.798570 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80a748-eb43-48fd-b960-a8c156ff9195 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:22.804587 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6cd8c-100b-417b-a7b4-f2edd2e513a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:22.805243 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6cd8c-100b-417b-a7b4-f2edd2e513a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:22.805523 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6cd8c-100b-417b-a7b4-f2edd2e513a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:22.805799 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6cd8c-100b-417b-a7b4-f2edd2e513a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:22.806476 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6cd8c-100b-417b-a7b4-f2edd2e513a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:22.808457 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6cd8c-100b-417b-a7b4-f2edd2e513a5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:22.842313 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1390/2802] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:50:22 2022] POST /identity/v3/projects => generated 386 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:50:22.848318 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249cf53a-29ef-465f-8d99-153e2c3dd8c5 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:22.859613 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249cf53a-29ef-465f-8d99-153e2c3dd8c5 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:22.931899 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc16c0-bc5e-4213-8fb5-1f18a8317ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:22.933277 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc16c0-bc5e-4213-8fb5-1f18a8317ffa admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:22.933922 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc16c0-bc5e-4213-8fb5-1f18a8317ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:22.941262 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc16c0-bc5e-4213-8fb5-1f18a8317ffa admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:22.941262 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc16c0-bc5e-4213-8fb5-1f18a8317ffa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:22.941262 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc16c0-bc5e-4213-8fb5-1f18a8317ffa admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:22.969320 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1413/2803] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:50:22 2022] POST /identity/v3/users => generated 294 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:50:22.981689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6cd8c-100b-417b-a7b4-f2edd2e513a5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:22.993955 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6cd8c-100b-417b-a7b4-f2edd2e513a5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:22.999435 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f473e-368d-44bb-8896-f78518ec9c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:23.000106 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f473e-368d-44bb-8896-f78518ec9c71 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:23.000418 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f473e-368d-44bb-8896-f78518ec9c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:23.000725 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f473e-368d-44bb-8896-f78518ec9c71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:23.001640 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f473e-368d-44bb-8896-f78518ec9c71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:23.007328 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f473e-368d-44bb-8896-f78518ec9c71 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:23.015725 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1391/2804] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:50:22 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:50:23.022111 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc16c0-bc5e-4213-8fb5-1f18a8317ffa admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:23.035896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc16c0-bc5e-4213-8fb5-1f18a8317ffa admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:23.042420 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ca25d-769c-4de5-bf40-f3bddebde44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:23.043082 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ca25d-769c-4de5-bf40-f3bddebde44d admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:23.043367 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ca25d-769c-4de5-bf40-f3bddebde44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:23.043643 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ca25d-769c-4de5-bf40-f3bddebde44d admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:23.044417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ca25d-769c-4de5-bf40-f3bddebde44d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:23.046082 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ca25d-769c-4de5-bf40-f3bddebde44d admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:23.059157 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1414/2805] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:50:23 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:50:23.064154 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f473e-368d-44bb-8896-f78518ec9c71 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:23.084554 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f473e-368d-44bb-8896-f78518ec9c71 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:23.086382 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3d455-d83c-4aaf-8d46-23d10a3a8e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:23.087629 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3d455-d83c-4aaf-8d46-23d10a3a8e1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:23.088143 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3d455-d83c-4aaf-8d46-23d10a3a8e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:23.089125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3d455-d83c-4aaf-8d46-23d10a3a8e1d admin admin] PATH_INFO: `/v3/projects/348fe3ede4ef4c9487bfab0394385f02/users/f487410e3a624da9b16f540539e04f3c/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:23.121444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d3d455-d83c-4aaf-8d46-23d10a3a8e1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=348fe3ede4ef4c9487bfab0394385f02, user_id=f487410e3a624da9b16f540539e04f3c, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:23.124171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d3d455-d83c-4aaf-8d46-23d10a3a8e1d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:23.139113 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1392/2806] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:50:23 2022] PUT /identity/v3/projects/348fe3ede4ef4c9487bfab0394385f02/users/f487410e3a624da9b16f540539e04f3c/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:23.211403 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25b4ca-7282-4822-9e67-feac3a8ef6a7 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:23.211678 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25b4ca-7282-4822-9e67-feac3a8ef6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:23.211981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25b4ca-7282-4822-9e67-feac3a8ef6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:23.240576 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:23.250807 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-ab25b4ca-7282-4822-9e67-feac3a8ef6a7 None None] MFA Rules not processed for user `f487410e3a624da9b16f540539e04f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:23.250807 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-ab25b4ca-7282-4822-9e67-feac3a8ef6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:23.337212 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1415/2807] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:50:23 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:50:23.337748 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3d455-d83c-4aaf-8d46-23d10a3a8e1d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:23.449708 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3d455-d83c-4aaf-8d46-23d10a3a8e1d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:23.454963 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7387bb8-9446-4e4b-93fc-6fd5fd6cb46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:23.455609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7387bb8-9446-4e4b-93fc-6fd5fd6cb46d admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:23.456067 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7387bb8-9446-4e4b-93fc-6fd5fd6cb46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:23.456435 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7387bb8-9446-4e4b-93fc-6fd5fd6cb46d admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:23.457140 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7387bb8-9446-4e4b-93fc-6fd5fd6cb46d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:23.459474 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7387bb8-9446-4e4b-93fc-6fd5fd6cb46d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:23.496520 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1393/2808] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:50:23 2022] POST /identity/v3/projects => generated 386 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:50:23.502193 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25b4ca-7282-4822-9e67-feac3a8ef6a7 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:23.519711 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25b4ca-7282-4822-9e67-feac3a8ef6a7 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:23.602395 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359237e-e62a-46cd-8443-27b5d14bc743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:23.603140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359237e-e62a-46cd-8443-27b5d14bc743 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:23.603316 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359237e-e62a-46cd-8443-27b5d14bc743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:23.603592 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359237e-e62a-46cd-8443-27b5d14bc743 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:23.604466 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359237e-e62a-46cd-8443-27b5d14bc743 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:23.606483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359237e-e62a-46cd-8443-27b5d14bc743 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:23.634290 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1416/2809] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:50:23 2022] POST /identity/v3/users => generated 294 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:50:23.639728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7387bb8-9446-4e4b-93fc-6fd5fd6cb46d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:23.653349 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7387bb8-9446-4e4b-93fc-6fd5fd6cb46d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:23.659211 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78309bf-ef66-4624-a0b2-3a08d5a73d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:23.659865 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78309bf-ef66-4624-a0b2-3a08d5a73d34 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:23.660163 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78309bf-ef66-4624-a0b2-3a08d5a73d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:23.660449 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78309bf-ef66-4624-a0b2-3a08d5a73d34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:23.661181 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78309bf-ef66-4624-a0b2-3a08d5a73d34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:23.662816 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78309bf-ef66-4624-a0b2-3a08d5a73d34 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:23.674871 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1394/2810] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:50:23 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:50:23.680386 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359237e-e62a-46cd-8443-27b5d14bc743 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:23.691358 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359237e-e62a-46cd-8443-27b5d14bc743 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:23.696385 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9861a9-6b56-469d-9606-ba58c00bb12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:23.697037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9861a9-6b56-469d-9606-ba58c00bb12d admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:23.697305 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9861a9-6b56-469d-9606-ba58c00bb12d admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:23.697645 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9861a9-6b56-469d-9606-ba58c00bb12d admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:23.698394 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9861a9-6b56-469d-9606-ba58c00bb12d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:23.700129 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9861a9-6b56-469d-9606-ba58c00bb12d admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:23.711814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1417/2811] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:50:23 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:50:23.717017 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78309bf-ef66-4624-a0b2-3a08d5a73d34 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:23.728211 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78309bf-ef66-4624-a0b2-3a08d5a73d34 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:23.733227 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2a3c7-9658-476c-bc67-fdc71d0c27a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:23.733915 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2a3c7-9658-476c-bc67-fdc71d0c27a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:23.734203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2a3c7-9658-476c-bc67-fdc71d0c27a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:23.734519 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2a3c7-9658-476c-bc67-fdc71d0c27a4 admin admin] PATH_INFO: `/v3/projects/5b0ca316716649b68177a2f9ba2b8877/users/d7bff56340cb4772902bf07c983e6612/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:23.752283 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2a3c7-9658-476c-bc67-fdc71d0c27a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b0ca316716649b68177a2f9ba2b8877, user_id=d7bff56340cb4772902bf07c983e6612, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:23.755025 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2a3c7-9658-476c-bc67-fdc71d0c27a4 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:23.771603 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1395/2812] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:50:23 2022] PUT /identity/v3/projects/5b0ca316716649b68177a2f9ba2b8877/users/d7bff56340cb4772902bf07c983e6612/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:23.782837 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f2457-562c-4f49-928b-bb99cf570e6c None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:23.783249 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f2457-562c-4f49-928b-bb99cf570e6c None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:23.783692 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f2457-562c-4f49-928b-bb99cf570e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:23.820340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-3d1f2457-562c-4f49-928b-bb99cf570e6c None None] MFA Rules not processed for user `d7bff56340cb4772902bf07c983e6612`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:23.821617 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-3d1f2457-562c-4f49-928b-bb99cf570e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:23.898575 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1418/2813] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:50:23 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:50:23.903652 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2a3c7-9658-476c-bc67-fdc71d0c27a4 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:23.920759 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:23.976345 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2a3c7-9658-476c-bc67-fdc71d0c27a4 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:23.982463 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882234c7-ff43-4491-b7c7-eb18dcdafd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:23.983139 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882234c7-ff43-4491-b7c7-eb18dcdafd8e admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:23.983443 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882234c7-ff43-4491-b7c7-eb18dcdafd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:23.983719 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882234c7-ff43-4491-b7c7-eb18dcdafd8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:23.984444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882234c7-ff43-4491-b7c7-eb18dcdafd8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:23.986459 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882234c7-ff43-4491-b7c7-eb18dcdafd8e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:24.020807 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1396/2814] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:50:23 2022] POST /identity/v3/projects => generated 386 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:50:24.025421 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f2457-562c-4f49-928b-bb99cf570e6c None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:24.037540 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f2457-562c-4f49-928b-bb99cf570e6c None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:24.047121 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:24.106186 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a170c0c-8ef6-4365-b5e8-a3b4b9837543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:24.106831 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a170c0c-8ef6-4365-b5e8-a3b4b9837543 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:24.107131 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a170c0c-8ef6-4365-b5e8-a3b4b9837543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:24.107429 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a170c0c-8ef6-4365-b5e8-a3b4b9837543 admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:24.108132 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a170c0c-8ef6-4365-b5e8-a3b4b9837543 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:24.110129 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a170c0c-8ef6-4365-b5e8-a3b4b9837543 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:24.136644 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1419/2815] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:50:24 2022] POST /identity/v3/users => generated 300 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:50:24.141138 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882234c7-ff43-4491-b7c7-eb18dcdafd8e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:24.153579 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882234c7-ff43-4491-b7c7-eb18dcdafd8e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:24.158746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f195993-5714-4eb0-b270-eea0c074d3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:24.159386 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f195993-5714-4eb0-b270-eea0c074d3be admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:24.159673 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f195993-5714-4eb0-b270-eea0c074d3be admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:24.160086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f195993-5714-4eb0-b270-eea0c074d3be admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:24.160695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f195993-5714-4eb0-b270-eea0c074d3be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:24.162387 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f195993-5714-4eb0-b270-eea0c074d3be admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:24.174472 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1397/2816] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:50:24 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:50:24.180140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a170c0c-8ef6-4365-b5e8-a3b4b9837543 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:24.193407 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a170c0c-8ef6-4365-b5e8-a3b4b9837543 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:24.199547 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ff234-87e0-4ce9-96da-c444046ecdb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:24.200319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ff234-87e0-4ce9-96da-c444046ecdb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:24.200597 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ff234-87e0-4ce9-96da-c444046ecdb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:24.200866 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ff234-87e0-4ce9-96da-c444046ecdb2 admin admin] PATH_INFO: `/v3/projects/836862499237452382b2e733bc63ee7e/users/ef1087e4a2df4b028efdd8da7ff03c3c/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:24.219378 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8ff234-87e0-4ce9-96da-c444046ecdb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=836862499237452382b2e733bc63ee7e, user_id=ef1087e4a2df4b028efdd8da7ff03c3c, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:24.222291 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8ff234-87e0-4ce9-96da-c444046ecdb2 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:24.245188 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1420/2817] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:50:24 2022] PUT /identity/v3/projects/836862499237452382b2e733bc63ee7e/users/ef1087e4a2df4b028efdd8da7ff03c3c/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:24.251088 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f195993-5714-4eb0-b270-eea0c074d3be admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:24.267340 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f195993-5714-4eb0-b270-eea0c074d3be admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:24.342774 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452df79-31e2-410a-9623-d6d65900f770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:24.343445 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452df79-31e2-410a-9623-d6d65900f770 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:24.343713 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452df79-31e2-410a-9623-d6d65900f770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:24.344053 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452df79-31e2-410a-9623-d6d65900f770 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:24.344805 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0452df79-31e2-410a-9623-d6d65900f770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:24.346574 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0452df79-31e2-410a-9623-d6d65900f770 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:24.360003 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1398/2818] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:50:24 2022] GET /identity/v3/roles => generated 1427 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:50:24.366234 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ff234-87e0-4ce9-96da-c444046ecdb2 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:24.381480 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ff234-87e0-4ce9-96da-c444046ecdb2 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:24.388188 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0307593-0569-447e-b99f-2f79788186f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:24.389181 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0307593-0569-447e-b99f-2f79788186f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:24.389579 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0307593-0569-447e-b99f-2f79788186f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:24.389973 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0307593-0569-447e-b99f-2f79788186f9 admin admin] PATH_INFO: `/v3/projects/836862499237452382b2e733bc63ee7e/users/ef1087e4a2df4b028efdd8da7ff03c3c/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:24.393040 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0307593-0569-447e-b99f-2f79788186f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=836862499237452382b2e733bc63ee7e, user_id=ef1087e4a2df4b028efdd8da7ff03c3c, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:24.397187 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0307593-0569-447e-b99f-2f79788186f9 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:24.413559 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1421/2819] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:50:24 2022] PUT /identity/v3/projects/836862499237452382b2e733bc63ee7e/users/ef1087e4a2df4b028efdd8da7ff03c3c/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:24.431155 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0487d45-aa9b-4e30-9ca6-ca8d552dd27a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:24.431602 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0487d45-aa9b-4e30-9ca6-ca8d552dd27a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:24.432105 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0487d45-aa9b-4e30-9ca6-ca8d552dd27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:24.481575 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b0487d45-aa9b-4e30-9ca6-ca8d552dd27a None None] MFA Rules not processed for user `ef1087e4a2df4b028efdd8da7ff03c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:24.483444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b0487d45-aa9b-4e30-9ca6-ca8d552dd27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:24.575806 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1399/2820] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:50:24 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:50:24.590581 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0307593-0569-447e-b99f-2f79788186f9 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:24.592514 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c0307593-0569-447e-b99f-2f79788186f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:24.666844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0307593-0569-447e-b99f-2f79788186f9 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:24.671150 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1938f6f-e81f-40b4-99c7-e9e0133fb51f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:24.672399 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1938f6f-e81f-40b4-99c7-e9e0133fb51f service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:24.672879 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1938f6f-e81f-40b4-99c7-e9e0133fb51f service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:24.673424 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1938f6f-e81f-40b4-99c7-e9e0133fb51f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:24.742730 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1938f6f-e81f-40b4-99c7-e9e0133fb51f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:24.744241 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1938f6f-e81f-40b4-99c7-e9e0133fb51f service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:24.754172 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1422/2821] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:50:24 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:50:26.361317 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0487d45-aa9b-4e30-9ca6-ca8d552dd27a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:26.363263 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b0487d45-aa9b-4e30-9ca6-ca8d552dd27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:26.379497 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0487d45-aa9b-4e30-9ca6-ca8d552dd27a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:26.383466 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2755153-c907-41e1-9825-3f3269fa2ce8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:26.384158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2755153-c907-41e1-9825-3f3269fa2ce8 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:26.384456 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2755153-c907-41e1-9825-3f3269fa2ce8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:26.384738 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2755153-c907-41e1-9825-3f3269fa2ce8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:26.391369 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:26.466362 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2755153-c907-41e1-9825-3f3269fa2ce8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:26.469205 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2755153-c907-41e1-9825-3f3269fa2ce8 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:26.486736 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1400/2822] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:50:26 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:50:27.207594 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1938f6f-e81f-40b4-99c7-e9e0133fb51f service cinder] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:27.218089 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1938f6f-e81f-40b4-99c7-e9e0133fb51f service cinder] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:27.222840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44465957-6316-4747-ad07-1d481de40b28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:27.223491 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44465957-6316-4747-ad07-1d481de40b28 service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:27.223776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44465957-6316-4747-ad07-1d481de40b28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:27.224105 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44465957-6316-4747-ad07-1d481de40b28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:27.234928 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44465957-6316-4747-ad07-1d481de40b28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:27.236447 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44465957-6316-4747-ad07-1d481de40b28 service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:27.247583 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1423/2823] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:50:27 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:50:31.282257 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2755153-c907-41e1-9825-3f3269fa2ce8 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:31.352600 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2755153-c907-41e1-9825-3f3269fa2ce8 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:31.358855 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8a5e8-6e2a-424a-a6e8-cb705112fb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:31.359531 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8a5e8-6e2a-424a-a6e8-cb705112fb67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:31.359823 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8a5e8-6e2a-424a-a6e8-cb705112fb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:31.360152 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8a5e8-6e2a-424a-a6e8-cb705112fb67 admin admin] PATH_INFO: `/v3/users/f487410e3a624da9b16f540539e04f3c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:31.361409 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba8a5e8-6e2a-424a-a6e8-cb705112fb67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f487410e3a624da9b16f540539e04f3c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:31.363378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba8a5e8-6e2a-424a-a6e8-cb705112fb67 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:31.417450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2ba8a5e8-6e2a-424a-a6e8-cb705112fb67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f487410e3a624da9b16f540539e04f3c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:31.424623 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2ba8a5e8-6e2a-424a-a6e8-cb705112fb67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f487410e3a624da9b16f540539e04f3c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:31.425371 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2ba8a5e8-6e2a-424a-a6e8-cb705112fb67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f487410e3a624da9b16f540539e04f3c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:31.442730 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2ba8a5e8-6e2a-424a-a6e8-cb705112fb67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f487410e3a624da9b16f540539e04f3c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:31.443557 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2ba8a5e8-6e2a-424a-a6e8-cb705112fb67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f487410e3a624da9b16f540539e04f3c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:31.459040 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1401/2824] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:50:31 2022] DELETE /identity/v3/users/f487410e3a624da9b16f540539e04f3c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:31.464742 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c11ac7-9fd5-4c25-85e8-7c9f8b595238 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:31.465313 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c11ac7-9fd5-4c25-85e8-7c9f8b595238 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:31.465313 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c11ac7-9fd5-4c25-85e8-7c9f8b595238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:31.474001 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:31.486742 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-25c11ac7-9fd5-4c25-85e8-7c9f8b595238 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:31.488027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-25c11ac7-9fd5-4c25-85e8-7c9f8b595238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:31.551580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1424/2825] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:50:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:50:31.564463 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8a5e8-6e2a-424a-a6e8-cb705112fb67 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:31.566559 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2ba8a5e8-6e2a-424a-a6e8-cb705112fb67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:31.610236 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:31.640132 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8a5e8-6e2a-424a-a6e8-cb705112fb67 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:31.644118 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12647f6d-0e62-4743-9645-60840af80ce4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:31.644763 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12647f6d-0e62-4743-9645-60840af80ce4 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:31.645051 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12647f6d-0e62-4743-9645-60840af80ce4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:31.645331 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12647f6d-0e62-4743-9645-60840af80ce4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:31.660670 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12647f6d-0e62-4743-9645-60840af80ce4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:31.660670 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12647f6d-0e62-4743-9645-60840af80ce4 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:31.668895 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1402/2826] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:50:31 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:50:31.994398 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c11ac7-9fd5-4c25-85e8-7c9f8b595238 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:31.996226 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-25c11ac7-9fd5-4c25-85e8-7c9f8b595238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:32.011776 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c11ac7-9fd5-4c25-85e8-7c9f8b595238 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:32.017247 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e38cfa-568b-446f-9d44-f62dc21538d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:32.017948 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e38cfa-568b-446f-9d44-f62dc21538d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:32.018236 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e38cfa-568b-446f-9d44-f62dc21538d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:32.018512 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e38cfa-568b-446f-9d44-f62dc21538d6 admin admin] PATH_INFO: `/v3/projects/348fe3ede4ef4c9487bfab0394385f02` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:32.019789 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e38cfa-568b-446f-9d44-f62dc21538d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=348fe3ede4ef4c9487bfab0394385f02)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:32.022082 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e38cfa-568b-446f-9d44-f62dc21538d6 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:32.089178 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b7e38cfa-568b-446f-9d44-f62dc21538d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '348fe3ede4ef4c9487bfab0394385f02', 'request_id': 'req-b7e38cfa-568b-446f-9d44-f62dc21538d6'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:32.099169 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b7e38cfa-568b-446f-9d44-f62dc21538d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '348fe3ede4ef4c9487bfab0394385f02', 'request_id': 'req-b7e38cfa-568b-446f-9d44-f62dc21538d6'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:32.105769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b7e38cfa-568b-446f-9d44-f62dc21538d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '348fe3ede4ef4c9487bfab0394385f02', 'request_id': 'req-b7e38cfa-568b-446f-9d44-f62dc21538d6'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:32.119142 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b7e38cfa-568b-446f-9d44-f62dc21538d6 admin admin] The token cache is being invalidate because project 348fe3ede4ef4c9487bfab0394385f02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:50:32.119551 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b7e38cfa-568b-446f-9d44-f62dc21538d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:32.120267 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-b7e38cfa-568b-446f-9d44-f62dc21538d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:32.121930 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1425/2827] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:50:31 2022] DELETE /identity/v3/projects/348fe3ede4ef4c9487bfab0394385f02 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:32.126510 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12647f6d-0e62-4743-9645-60840af80ce4 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:32.138639 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12647f6d-0e62-4743-9645-60840af80ce4 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:32.214026 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78cacdc-0063-4d87-a379-ce639ec40faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:32.214681 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78cacdc-0063-4d87-a379-ce639ec40faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:32.214973 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78cacdc-0063-4d87-a379-ce639ec40faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:32.215257 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78cacdc-0063-4d87-a379-ce639ec40faa admin admin] PATH_INFO: `/v3/users/d7bff56340cb4772902bf07c983e6612` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:32.216621 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78cacdc-0063-4d87-a379-ce639ec40faa admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7bff56340cb4772902bf07c983e6612)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:32.218668 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78cacdc-0063-4d87-a379-ce639ec40faa admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:32.285386 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b78cacdc-0063-4d87-a379-ce639ec40faa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7bff56340cb4772902bf07c983e6612'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:32.295298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b78cacdc-0063-4d87-a379-ce639ec40faa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7bff56340cb4772902bf07c983e6612'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:32.296129 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b78cacdc-0063-4d87-a379-ce639ec40faa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7bff56340cb4772902bf07c983e6612'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:32.309580 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b78cacdc-0063-4d87-a379-ce639ec40faa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7bff56340cb4772902bf07c983e6612'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:32.310365 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b78cacdc-0063-4d87-a379-ce639ec40faa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7bff56340cb4772902bf07c983e6612'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:32.326540 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1403/2828] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:50:32 2022] DELETE /identity/v3/users/d7bff56340cb4772902bf07c983e6612 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:32.627047 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e38cfa-568b-446f-9d44-f62dc21538d6 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:32.628871 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-b7e38cfa-568b-446f-9d44-f62dc21538d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:32.647494 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e38cfa-568b-446f-9d44-f62dc21538d6 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:32.718489 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e13100-ca21-4091-8359-9f13e362d260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:32.719175 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e13100-ca21-4091-8359-9f13e362d260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:32.719450 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e13100-ca21-4091-8359-9f13e362d260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:32.719728 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e13100-ca21-4091-8359-9f13e362d260 admin admin] PATH_INFO: `/v3/projects/5b0ca316716649b68177a2f9ba2b8877` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:32.721018 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e13100-ca21-4091-8359-9f13e362d260 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b0ca316716649b68177a2f9ba2b8877)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:32.722982 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e13100-ca21-4091-8359-9f13e362d260 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:32.767735 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:32.785870 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f3e13100-ca21-4091-8359-9f13e362d260 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b0ca316716649b68177a2f9ba2b8877', 'request_id': 'req-f3e13100-ca21-4091-8359-9f13e362d260'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:32.795451 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f3e13100-ca21-4091-8359-9f13e362d260 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b0ca316716649b68177a2f9ba2b8877', 'request_id': 'req-f3e13100-ca21-4091-8359-9f13e362d260'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:32.802770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f3e13100-ca21-4091-8359-9f13e362d260 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b0ca316716649b68177a2f9ba2b8877', 'request_id': 'req-f3e13100-ca21-4091-8359-9f13e362d260'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:32.825532 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f3e13100-ca21-4091-8359-9f13e362d260 admin admin] The token cache is being invalidate because project 5b0ca316716649b68177a2f9ba2b8877 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:50:32.825894 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f3e13100-ca21-4091-8359-9f13e362d260 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:32.826901 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f3e13100-ca21-4091-8359-9f13e362d260 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:32.828276 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1426/2829] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:50:32 2022] DELETE /identity/v3/projects/5b0ca316716649b68177a2f9ba2b8877 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:32.833060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78cacdc-0063-4d87-a379-ce639ec40faa admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:32.834629 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b78cacdc-0063-4d87-a379-ce639ec40faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:32.849344 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78cacdc-0063-4d87-a379-ce639ec40faa admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:32.902509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:32.919421 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7fbc3a-5111-42ea-8f50-9deee92ca604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:32.920158 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fbc3a-5111-42ea-8f50-9deee92ca604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:32.920447 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fbc3a-5111-42ea-8f50-9deee92ca604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:32.920728 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fbc3a-5111-42ea-8f50-9deee92ca604 admin admin] PATH_INFO: `/v3/users/ef1087e4a2df4b028efdd8da7ff03c3c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:32.921994 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7fbc3a-5111-42ea-8f50-9deee92ca604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef1087e4a2df4b028efdd8da7ff03c3c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:32.924004 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7fbc3a-5111-42ea-8f50-9deee92ca604 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:32.982369 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7b7fbc3a-5111-42ea-8f50-9deee92ca604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef1087e4a2df4b028efdd8da7ff03c3c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:32.990496 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7b7fbc3a-5111-42ea-8f50-9deee92ca604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef1087e4a2df4b028efdd8da7ff03c3c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:32.991293 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7b7fbc3a-5111-42ea-8f50-9deee92ca604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef1087e4a2df4b028efdd8da7ff03c3c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:33.007622 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7b7fbc3a-5111-42ea-8f50-9deee92ca604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef1087e4a2df4b028efdd8da7ff03c3c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:33.008435 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7b7fbc3a-5111-42ea-8f50-9deee92ca604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef1087e4a2df4b028efdd8da7ff03c3c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:33.025242 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1404/2830] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:50:32 2022] DELETE /identity/v3/users/ef1087e4a2df4b028efdd8da7ff03c3c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:33.358909 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e13100-ca21-4091-8359-9f13e362d260 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:33.360533 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-f3e13100-ca21-4091-8359-9f13e362d260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:33.376312 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e13100-ca21-4091-8359-9f13e362d260 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:33.442501 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8138fba0-b261-48ac-974e-cb35bfc9a36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:33.443042 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138fba0-b261-48ac-974e-cb35bfc9a36f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:33.443331 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138fba0-b261-48ac-974e-cb35bfc9a36f admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:33.443590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138fba0-b261-48ac-974e-cb35bfc9a36f admin admin] PATH_INFO: `/v3/projects/836862499237452382b2e733bc63ee7e` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:33.444866 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8138fba0-b261-48ac-974e-cb35bfc9a36f admin admin] RBAC: Authorizing `identity:delete_project(project_id=836862499237452382b2e733bc63ee7e)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:33.446812 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8138fba0-b261-48ac-974e-cb35bfc9a36f admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:33.500176 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-8138fba0-b261-48ac-974e-cb35bfc9a36f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '836862499237452382b2e733bc63ee7e', 'request_id': 'req-8138fba0-b261-48ac-974e-cb35bfc9a36f'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:33.510174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-8138fba0-b261-48ac-974e-cb35bfc9a36f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '836862499237452382b2e733bc63ee7e', 'request_id': 'req-8138fba0-b261-48ac-974e-cb35bfc9a36f'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:33.517697 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-8138fba0-b261-48ac-974e-cb35bfc9a36f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '836862499237452382b2e733bc63ee7e', 'request_id': 'req-8138fba0-b261-48ac-974e-cb35bfc9a36f'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:33.528610 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-8138fba0-b261-48ac-974e-cb35bfc9a36f admin admin] The token cache is being invalidate because project 836862499237452382b2e733bc63ee7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:50:33.528924 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-8138fba0-b261-48ac-974e-cb35bfc9a36f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:33.529616 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-8138fba0-b261-48ac-974e-cb35bfc9a36f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:33.531380 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1427/2831] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:50:33 2022] DELETE /identity/v3/projects/836862499237452382b2e733bc63ee7e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:33.537162 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5430869-005b-4135-bb87-c52015509be6 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:33.537756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5430869-005b-4135-bb87-c52015509be6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:33.537756 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5430869-005b-4135-bb87-c52015509be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:33.561975 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f5430869-005b-4135-bb87-c52015509be6 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:33.563266 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f5430869-005b-4135-bb87-c52015509be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:33.635401 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1405/2832] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:50:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:50:33.645130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55ba97-94aa-410b-8389-3ddf40ffe6f9 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:33.646099 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55ba97-94aa-410b-8389-3ddf40ffe6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:33.646099 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55ba97-94aa-410b-8389-3ddf40ffe6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:33.675831 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-5c55ba97-94aa-410b-8389-3ddf40ffe6f9 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:33.677625 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-5c55ba97-94aa-410b-8389-3ddf40ffe6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:33.688212 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1428/2833] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:50:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:50:33.693298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5430869-005b-4135-bb87-c52015509be6 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:33.704691 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5430869-005b-4135-bb87-c52015509be6 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:33.712532 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbdeca-389b-47bf-acdc-d5882dd3ed3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:33.713278 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbdeca-389b-47bf-acdc-d5882dd3ed3c admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:33.713926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbdeca-389b-47bf-acdc-d5882dd3ed3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:33.714184 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbdeca-389b-47bf-acdc-d5882dd3ed3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:33.714890 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbdeca-389b-47bf-acdc-d5882dd3ed3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:50:33.715214 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbdeca-389b-47bf-acdc-d5882dd3ed3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:33.716879 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbdeca-389b-47bf-acdc-d5882dd3ed3c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:33.730036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1406/2834] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:50:33 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 10:50:33.736176 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c55ba97-94aa-410b-8389-3ddf40ffe6f9 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:33.747365 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c55ba97-94aa-410b-8389-3ddf40ffe6f9 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:33.754036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b2a4b-deeb-4ab0-8284-1b24ffaba1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:33.756438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b2a4b-deeb-4ab0-8284-1b24ffaba1dc admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:33.756438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b2a4b-deeb-4ab0-8284-1b24ffaba1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:33.756438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b2a4b-deeb-4ab0-8284-1b24ffaba1dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:33.756438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b2a4b-deeb-4ab0-8284-1b24ffaba1dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:33.757971 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b2a4b-deeb-4ab0-8284-1b24ffaba1dc admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:33.793060 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1429/2835] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:50:33 2022] POST /identity/v3/projects => generated 374 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:50:33.799222 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbdeca-389b-47bf-acdc-d5882dd3ed3c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:33.814737 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbdeca-389b-47bf-acdc-d5882dd3ed3c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:33.901307 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:33.914171 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dbf41e-8202-418b-a13c-16bbb2a3a6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:33.914995 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dbf41e-8202-418b-a13c-16bbb2a3a6bd admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:33.915396 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dbf41e-8202-418b-a13c-16bbb2a3a6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:33.915778 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dbf41e-8202-418b-a13c-16bbb2a3a6bd admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:33.916928 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dbf41e-8202-418b-a13c-16bbb2a3a6bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:33.919785 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dbf41e-8202-418b-a13c-16bbb2a3a6bd admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:33.952781 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1407/2836] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:50:33 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:50:33.957576 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b2a4b-deeb-4ab0-8284-1b24ffaba1dc admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:33.970189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b2a4b-deeb-4ab0-8284-1b24ffaba1dc admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:33.976139 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278acad2-45fd-4370-ae47-0c96d790ff9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:33.976795 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278acad2-45fd-4370-ae47-0c96d790ff9c admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:33.977073 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278acad2-45fd-4370-ae47-0c96d790ff9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:33.977347 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278acad2-45fd-4370-ae47-0c96d790ff9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:33.978067 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278acad2-45fd-4370-ae47-0c96d790ff9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:33.979826 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278acad2-45fd-4370-ae47-0c96d790ff9c admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:33.991478 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1430/2837] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:50:33 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:50:33.996490 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dbf41e-8202-418b-a13c-16bbb2a3a6bd admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:34.010312 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dbf41e-8202-418b-a13c-16bbb2a3a6bd admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:34.015844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19471c5b-b6de-4212-8e4e-2c95f41a1825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:34.016554 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19471c5b-b6de-4212-8e4e-2c95f41a1825 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:34.016846 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19471c5b-b6de-4212-8e4e-2c95f41a1825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:34.017136 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19471c5b-b6de-4212-8e4e-2c95f41a1825 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:34.017865 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19471c5b-b6de-4212-8e4e-2c95f41a1825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:34.019527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19471c5b-b6de-4212-8e4e-2c95f41a1825 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:34.034105 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1408/2838] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:50:33 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:50:34.039523 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278acad2-45fd-4370-ae47-0c96d790ff9c admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:34.053340 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278acad2-45fd-4370-ae47-0c96d790ff9c admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:34.062265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1ed10-1907-4f27-aa7a-b794b1164a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:34.063260 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1ed10-1907-4f27-aa7a-b794b1164a31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:34.063615 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1ed10-1907-4f27-aa7a-b794b1164a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:34.063996 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1ed10-1907-4f27-aa7a-b794b1164a31 admin admin] PATH_INFO: `/v3/projects/4dbc439ba67c4c87b69069bd6a79f077/users/c3466834cbe847b7a50d1b07b468629a/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:34.068196 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:34.083490 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce1ed10-1907-4f27-aa7a-b794b1164a31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dbc439ba67c4c87b69069bd6a79f077, user_id=c3466834cbe847b7a50d1b07b468629a, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:34.086571 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce1ed10-1907-4f27-aa7a-b794b1164a31 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:34.104664 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1431/2839] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:50:34 2022] PUT /identity/v3/projects/4dbc439ba67c4c87b69069bd6a79f077/users/c3466834cbe847b7a50d1b07b468629a/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:34.119302 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1d872-03eb-4756-add8-3ae56cc1365e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:34.119702 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1d872-03eb-4756-add8-3ae56cc1365e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:34.120014 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1d872-03eb-4756-add8-3ae56cc1365e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:34.155647 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e2b1d872-03eb-4756-add8-3ae56cc1365e None None] MFA Rules not processed for user `c3466834cbe847b7a50d1b07b468629a`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:34.157388 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e2b1d872-03eb-4756-add8-3ae56cc1365e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:34.245992 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1409/2840] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:50:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:50:34.258765 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1ed10-1907-4f27-aa7a-b794b1164a31 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:34.260862 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-9ce1ed10-1907-4f27-aa7a-b794b1164a31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:34.332041 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1ed10-1907-4f27-aa7a-b794b1164a31 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:34.335970 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccede4-f46d-4d33-8c88-f965e0d78fac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:34.336650 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccede4-f46d-4d33-8c88-f965e0d78fac service cinder] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:34.336933 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccede4-f46d-4d33-8c88-f965e0d78fac service cinder] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:34.337214 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccede4-f46d-4d33-8c88-f965e0d78fac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:34.348207 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ccede4-f46d-4d33-8c88-f965e0d78fac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:34.349702 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ccede4-f46d-4d33-8c88-f965e0d78fac service cinder] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:34.359803 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1432/2841] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:50:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:50:34.387407 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1d872-03eb-4756-add8-3ae56cc1365e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:34.460876 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1d872-03eb-4756-add8-3ae56cc1365e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:34.467202 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ad9ba-082e-4be0-b935-a59faed2e6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:34.467886 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ad9ba-082e-4be0-b935-a59faed2e6f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:34.468203 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ad9ba-082e-4be0-b935-a59faed2e6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:34.468482 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ad9ba-082e-4be0-b935-a59faed2e6f9 admin admin] PATH_INFO: `/v3/users/c3466834cbe847b7a50d1b07b468629a` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:34.469845 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023ad9ba-082e-4be0-b935-a59faed2e6f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3466834cbe847b7a50d1b07b468629a)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:34.471932 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023ad9ba-082e-4be0-b935-a59faed2e6f9 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:34.533602 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-023ad9ba-082e-4be0-b935-a59faed2e6f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3466834cbe847b7a50d1b07b468629a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:34.542072 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-023ad9ba-082e-4be0-b935-a59faed2e6f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3466834cbe847b7a50d1b07b468629a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:34.543086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-023ad9ba-082e-4be0-b935-a59faed2e6f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3466834cbe847b7a50d1b07b468629a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:34.553960 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:34.563439 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-023ad9ba-082e-4be0-b935-a59faed2e6f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3466834cbe847b7a50d1b07b468629a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:34.564681 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-023ad9ba-082e-4be0-b935-a59faed2e6f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3466834cbe847b7a50d1b07b468629a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:34.585049 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1410/2842] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:50:34 2022] DELETE /identity/v3/users/c3466834cbe847b7a50d1b07b468629a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:34.590644 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7ed45-5203-4733-86b3-0dd8cb40cdd8 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:34.590983 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7ed45-5203-4733-86b3-0dd8cb40cdd8 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:34.591238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7ed45-5203-4733-86b3-0dd8cb40cdd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:34.619608 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-48a7ed45-5203-4733-86b3-0dd8cb40cdd8 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:34.620981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-48a7ed45-5203-4733-86b3-0dd8cb40cdd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:34.691131 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1433/2843] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:50:34 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 10:50:34.702103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ad9ba-082e-4be0-b935-a59faed2e6f9 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:34.703584 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-023ad9ba-082e-4be0-b935-a59faed2e6f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:34.767649 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ad9ba-082e-4be0-b935-a59faed2e6f9 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:34.771653 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b35bf1-f183-4007-96a1-187a9f98af9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:34.772411 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b35bf1-f183-4007-96a1-187a9f98af9a service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:34.772695 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b35bf1-f183-4007-96a1-187a9f98af9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:34.772971 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b35bf1-f183-4007-96a1-187a9f98af9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:34.783217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b35bf1-f183-4007-96a1-187a9f98af9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:34.784935 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b35bf1-f183-4007-96a1-187a9f98af9a service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:34.796001 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1411/2844] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:50:34 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:50:35.103607 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7ed45-5203-4733-86b3-0dd8cb40cdd8 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:35.105239 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-48a7ed45-5203-4733-86b3-0dd8cb40cdd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:35.118118 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7ed45-5203-4733-86b3-0dd8cb40cdd8 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:35.123824 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd136d-0645-40bc-b241-2b05dfff79f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:35.125230 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd136d-0645-40bc-b241-2b05dfff79f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:35.125472 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd136d-0645-40bc-b241-2b05dfff79f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:35.125890 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd136d-0645-40bc-b241-2b05dfff79f1 admin admin] PATH_INFO: `/v3/projects/4dbc439ba67c4c87b69069bd6a79f077` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:35.127302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd136d-0645-40bc-b241-2b05dfff79f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dbc439ba67c4c87b69069bd6a79f077)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:35.130250 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd136d-0645-40bc-b241-2b05dfff79f1 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:35.188665 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f4cd136d-0645-40bc-b241-2b05dfff79f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dbc439ba67c4c87b69069bd6a79f077', 'request_id': 'req-f4cd136d-0645-40bc-b241-2b05dfff79f1'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:35.198180 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f4cd136d-0645-40bc-b241-2b05dfff79f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dbc439ba67c4c87b69069bd6a79f077', 'request_id': 'req-f4cd136d-0645-40bc-b241-2b05dfff79f1'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:35.205660 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f4cd136d-0645-40bc-b241-2b05dfff79f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dbc439ba67c4c87b69069bd6a79f077', 'request_id': 'req-f4cd136d-0645-40bc-b241-2b05dfff79f1'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:35.219185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f4cd136d-0645-40bc-b241-2b05dfff79f1 admin admin] The token cache is being invalidate because project 4dbc439ba67c4c87b69069bd6a79f077 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:50:35.219618 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f4cd136d-0645-40bc-b241-2b05dfff79f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:35.220386 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-f4cd136d-0645-40bc-b241-2b05dfff79f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:50:35.221974 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1434/2845] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:50:35 2022] DELETE /identity/v3/projects/4dbc439ba67c4c87b69069bd6a79f077 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 10:50:35.227514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4c2ba-d78d-4f8d-b875-00c90e68b3b7 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:35.227796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4c2ba-d78d-4f8d-b875-00c90e68b3b7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:35.228122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4c2ba-d78d-4f8d-b875-00c90e68b3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:35.249839 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-13e4c2ba-d78d-4f8d-b875-00c90e68b3b7 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:35.251264 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-13e4c2ba-d78d-4f8d-b875-00c90e68b3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:35.321716 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1412/2846] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:50:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:50:35.330418 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aca7e5-c1ad-471a-bc07-2cc38f54e075 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:35.330709 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aca7e5-c1ad-471a-bc07-2cc38f54e075 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:35.330994 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aca7e5-c1ad-471a-bc07-2cc38f54e075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:35.352797 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-66aca7e5-c1ad-471a-bc07-2cc38f54e075 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:35.354200 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-66aca7e5-c1ad-471a-bc07-2cc38f54e075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:35.363215 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1435/2847] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:50:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:50:35.368245 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4c2ba-d78d-4f8d-b875-00c90e68b3b7 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:35.379382 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4c2ba-d78d-4f8d-b875-00c90e68b3b7 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:35.384390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383547a9-b319-48a6-8399-5bdae92aca2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:35.386411 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383547a9-b319-48a6-8399-5bdae92aca2d admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:35.386720 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383547a9-b319-48a6-8399-5bdae92aca2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:35.387003 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383547a9-b319-48a6-8399-5bdae92aca2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:35.388424 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383547a9-b319-48a6-8399-5bdae92aca2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:50:35.388768 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383547a9-b319-48a6-8399-5bdae92aca2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:35.390804 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383547a9-b319-48a6-8399-5bdae92aca2d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:35.405798 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1413/2848] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:50:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:50:35.410706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aca7e5-c1ad-471a-bc07-2cc38f54e075 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:35.417278 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:35.421785 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aca7e5-c1ad-471a-bc07-2cc38f54e075 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:35.429873 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30dfe0-c009-4652-8b96-eea222e2a457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:35.430651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30dfe0-c009-4652-8b96-eea222e2a457 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:35.431189 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30dfe0-c009-4652-8b96-eea222e2a457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:35.431699 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30dfe0-c009-4652-8b96-eea222e2a457 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:35.432585 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30dfe0-c009-4652-8b96-eea222e2a457 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:35.434840 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30dfe0-c009-4652-8b96-eea222e2a457 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:35.467810 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1436/2849] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:50:35 2022] POST /identity/v3/projects => generated 386 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:50:35.472483 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383547a9-b319-48a6-8399-5bdae92aca2d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:35.485503 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383547a9-b319-48a6-8399-5bdae92aca2d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:35.554265 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c57e22-be38-411b-9e51-ce2d8d9c994c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:35.555683 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c57e22-be38-411b-9e51-ce2d8d9c994c admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:35.556009 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c57e22-be38-411b-9e51-ce2d8d9c994c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:35.556324 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c57e22-be38-411b-9e51-ce2d8d9c994c admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:35.557030 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c57e22-be38-411b-9e51-ce2d8d9c994c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:35.559086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c57e22-be38-411b-9e51-ce2d8d9c994c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:35.591262 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1414/2850] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:50:35 2022] POST /identity/v3/users => generated 294 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:50:35.596014 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30dfe0-c009-4652-8b96-eea222e2a457 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:35.608263 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30dfe0-c009-4652-8b96-eea222e2a457 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:35.614447 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ce252-6e9d-4b31-bb9c-8a7ceacdb4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:35.614800 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ce252-6e9d-4b31-bb9c-8a7ceacdb4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:35.615234 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ce252-6e9d-4b31-bb9c-8a7ceacdb4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:35.615536 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ce252-6e9d-4b31-bb9c-8a7ceacdb4e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:35.616335 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ce252-6e9d-4b31-bb9c-8a7ceacdb4e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:35.617994 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ce252-6e9d-4b31-bb9c-8a7ceacdb4e7 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:35.629577 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1437/2851] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:50:35 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:50:35.634459 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c57e22-be38-411b-9e51-ce2d8d9c994c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:35.640148 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:35.647746 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c57e22-be38-411b-9e51-ce2d8d9c994c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:35.655373 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2558ee7-a3bc-4872-993e-e8aad305e9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:35.656065 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2558ee7-a3bc-4872-993e-e8aad305e9b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:35.656358 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2558ee7-a3bc-4872-993e-e8aad305e9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:35.656635 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2558ee7-a3bc-4872-993e-e8aad305e9b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:35.657365 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2558ee7-a3bc-4872-993e-e8aad305e9b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:35.659024 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2558ee7-a3bc-4872-993e-e8aad305e9b6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:35.673770 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1415/2852] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:50:35 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 10:50:35.679500 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ce252-6e9d-4b31-bb9c-8a7ceacdb4e7 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:35.691620 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ce252-6e9d-4b31-bb9c-8a7ceacdb4e7 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:35.700847 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79366f62-e671-4627-881c-27658ec85d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:35.701910 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79366f62-e671-4627-881c-27658ec85d8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:35.702240 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79366f62-e671-4627-881c-27658ec85d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:35.702721 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79366f62-e671-4627-881c-27658ec85d8b admin admin] PATH_INFO: `/v3/projects/9a6e2ee8a698471faedbb97fb2b936cd/users/26ad0a8e94ab4bcfa8b01dc16ad08b0f/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:35.721749 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79366f62-e671-4627-881c-27658ec85d8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a6e2ee8a698471faedbb97fb2b936cd, user_id=26ad0a8e94ab4bcfa8b01dc16ad08b0f, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:35.724507 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79366f62-e671-4627-881c-27658ec85d8b admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:35.742115 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1438/2853] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:50:35 2022] PUT /identity/v3/projects/9a6e2ee8a698471faedbb97fb2b936cd/users/26ad0a8e94ab4bcfa8b01dc16ad08b0f/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:35.747803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a103e0b-882d-416c-b724-d2b85699b617 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:35.748153 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a103e0b-882d-416c-b724-d2b85699b617 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:35.748440 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a103e0b-882d-416c-b724-d2b85699b617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:35.768968 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8a103e0b-882d-416c-b724-d2b85699b617 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:35.770272 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8a103e0b-882d-416c-b724-d2b85699b617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:35.837837 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1416/2854] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:50:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:50:35.848419 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79366f62-e671-4627-881c-27658ec85d8b admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:35.850413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-79366f62-e671-4627-881c-27658ec85d8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:35.914810 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79366f62-e671-4627-881c-27658ec85d8b admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:35.919176 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400221d-f0fb-4274-98d5-685763605f4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:35.919818 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400221d-f0fb-4274-98d5-685763605f4c service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:35.920143 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400221d-f0fb-4274-98d5-685763605f4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:35.920401 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400221d-f0fb-4274-98d5-685763605f4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:35.930600 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c400221d-f0fb-4274-98d5-685763605f4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:35.932329 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c400221d-f0fb-4274-98d5-685763605f4c service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:35.942319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1439/2855] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:50:35 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:50:40.551258 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a188466-6631-43de-84b4-ae3e842b4f26 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:40.551927 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a188466-6631-43de-84b4-ae3e842b4f26 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:40.551927 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a188466-6631-43de-84b4-ae3e842b4f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:40.586280 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7a188466-6631-43de-84b4-ae3e842b4f26 None None] MFA Rules not processed for user `26ad0a8e94ab4bcfa8b01dc16ad08b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:40.587071 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7a188466-6631-43de-84b4-ae3e842b4f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:40.667367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1417/2856] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:50:40 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:50:40.672466 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400221d-f0fb-4274-98d5-685763605f4c service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:40.683369 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400221d-f0fb-4274-98d5-685763605f4c service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:40.689268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323d483-76ce-4b63-8de0-4932f64e2433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:40.689825 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323d483-76ce-4b63-8de0-4932f64e2433 admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:40.690107 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323d483-76ce-4b63-8de0-4932f64e2433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:40.690383 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323d483-76ce-4b63-8de0-4932f64e2433 admin admin] PATH_INFO: `/v3/projects` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:40.691060 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a323d483-76ce-4b63-8de0-4932f64e2433 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:40.693058 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a323d483-76ce-4b63-8de0-4932f64e2433 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:40.725177 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1440/2857] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:50:40 2022] POST /identity/v3/projects => generated 386 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:50:40.729830 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a188466-6631-43de-84b4-ae3e842b4f26 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:40.742936 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a188466-6631-43de-84b4-ae3e842b4f26 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:40.806281 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af9dd8-2d35-4795-9bc8-b5ad300f12de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:40.806969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af9dd8-2d35-4795-9bc8-b5ad300f12de admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:40.807252 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af9dd8-2d35-4795-9bc8-b5ad300f12de admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:40.807529 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af9dd8-2d35-4795-9bc8-b5ad300f12de admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:40.808325 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af9dd8-2d35-4795-9bc8-b5ad300f12de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:40.810237 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af9dd8-2d35-4795-9bc8-b5ad300f12de admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:40.835800 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1418/2858] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:50:40 2022] POST /identity/v3/users => generated 300 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:50:40.840578 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323d483-76ce-4b63-8de0-4932f64e2433 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:40.852657 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323d483-76ce-4b63-8de0-4932f64e2433 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:40.858703 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3de1c-4570-4dfc-9a5c-b836eb3ce43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:40.859352 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3de1c-4570-4dfc-9a5c-b836eb3ce43f admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:40.859630 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3de1c-4570-4dfc-9a5c-b836eb3ce43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:40.859904 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3de1c-4570-4dfc-9a5c-b836eb3ce43f admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:40.860647 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a3de1c-4570-4dfc-9a5c-b836eb3ce43f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:40.862293 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a3de1c-4570-4dfc-9a5c-b836eb3ce43f admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:40.874293 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1441/2859] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:50:40 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:50:40.879061 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af9dd8-2d35-4795-9bc8-b5ad300f12de admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:40.891035 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af9dd8-2d35-4795-9bc8-b5ad300f12de admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:40.898351 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed56d7-d46a-41a8-8951-d14fc33d1a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:40.899080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed56d7-d46a-41a8-8951-d14fc33d1a8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:40.899356 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed56d7-d46a-41a8-8951-d14fc33d1a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:40.899627 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed56d7-d46a-41a8-8951-d14fc33d1a8a admin admin] PATH_INFO: `/v3/projects/6afb923fc7f545e0a59aa9007caea945/users/68eac32080534163ad17e3a9e79cf3ef/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:40.911095 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:40.921118 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed56d7-d46a-41a8-8951-d14fc33d1a8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6afb923fc7f545e0a59aa9007caea945, user_id=68eac32080534163ad17e3a9e79cf3ef, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:40.924321 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed56d7-d46a-41a8-8951-d14fc33d1a8a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:40.939434 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1419/2860] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:50:40 2022] PUT /identity/v3/projects/6afb923fc7f545e0a59aa9007caea945/users/68eac32080534163ad17e3a9e79cf3ef/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:40.943978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3de1c-4570-4dfc-9a5c-b836eb3ce43f admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:40.954687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3de1c-4570-4dfc-9a5c-b836eb3ce43f admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:41.021130 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41569d6c-1c1c-4d3c-918e-5c18a26a8e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:41.021649 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41569d6c-1c1c-4d3c-918e-5c18a26a8e0a admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:41.022002 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41569d6c-1c1c-4d3c-918e-5c18a26a8e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:41.022188 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41569d6c-1c1c-4d3c-918e-5c18a26a8e0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:41.023108 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41569d6c-1c1c-4d3c-918e-5c18a26a8e0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:41.025332 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41569d6c-1c1c-4d3c-918e-5c18a26a8e0a admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:41.039238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1442/2861] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:50:40 2022] GET /identity/v3/roles => generated 1427 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:50:41.044816 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed56d7-d46a-41a8-8951-d14fc33d1a8a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:41.057333 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed56d7-d46a-41a8-8951-d14fc33d1a8a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:41.062799 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab52913-c033-45e1-8c74-a81bd2ebd4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:41.063731 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab52913-c033-45e1-8c74-a81bd2ebd4f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:41.064247 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab52913-c033-45e1-8c74-a81bd2ebd4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:41.064522 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab52913-c033-45e1-8c74-a81bd2ebd4f6 admin admin] PATH_INFO: `/v3/projects/6afb923fc7f545e0a59aa9007caea945/users/68eac32080534163ad17e3a9e79cf3ef/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:41.066757 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab52913-c033-45e1-8c74-a81bd2ebd4f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6afb923fc7f545e0a59aa9007caea945, user_id=68eac32080534163ad17e3a9e79cf3ef, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:41.070021 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab52913-c033-45e1-8c74-a81bd2ebd4f6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:41.090864 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1420/2862] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:50:41 2022] PUT /identity/v3/projects/6afb923fc7f545e0a59aa9007caea945/users/68eac32080534163ad17e3a9e79cf3ef/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:50:46.075820 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57ca84-7965-4f95-9a10-f43062954278 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:46.076521 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57ca84-7965-4f95-9a10-f43062954278 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:46.079061 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57ca84-7965-4f95-9a10-f43062954278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:46.110938 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:46.114947 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-cc57ca84-7965-4f95-9a10-f43062954278 None None] MFA Rules not processed for user `68eac32080534163ad17e3a9e79cf3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:46.117613 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-cc57ca84-7965-4f95-9a10-f43062954278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:46.206972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1443/2863] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:50:46 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:50:46.224723 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab52913-c033-45e1-8c74-a81bd2ebd4f6 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:46.226712 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3ab52913-c033-45e1-8c74-a81bd2ebd4f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:46.297065 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab52913-c033-45e1-8c74-a81bd2ebd4f6 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:46.301529 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f67de-2e85-4423-a418-a649e98aea4f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b31c86bc9334d2b8acc35a035f90442', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:46.302193 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f67de-2e85-4423-a418-a649e98aea4f service glance] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:46.302478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f67de-2e85-4423-a418-a649e98aea4f service glance] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:46.302782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f67de-2e85-4423-a418-a649e98aea4f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:46.370200 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f67de-2e85-4423-a418-a649e98aea4f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:46.371570 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f67de-2e85-4423-a418-a649e98aea4f service glance] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:46.382509 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1421/2864] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:50:46 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:50:46.516193 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2ce10-d27f-4f97-beb7-dc0f94bcb8ff None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:46.516512 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2ce10-d27f-4f97-beb7-dc0f94bcb8ff None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:46.516814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2ce10-d27f-4f97-beb7-dc0f94bcb8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:46.542982 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-1ce2ce10-d27f-4f97-beb7-dc0f94bcb8ff None None] MFA Rules not processed for user `efe6e59228bd49bfbff25a11eb682426`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:46.544852 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-1ce2ce10-d27f-4f97-beb7-dc0f94bcb8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:46.629980 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1444/2865] 10.222.0.121 () {58 vars in 1014 bytes} [Thu Aug 4 10:50:46 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:50:46.694751 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f67de-2e85-4423-a418-a649e98aea4f service glance] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:46.696577 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-487f67de-2e85-4423-a418-a649e98aea4f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:46.776404 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f67de-2e85-4423-a418-a649e98aea4f service glance] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:46.782453 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35bfe8-628b-438a-9698-40bad0ad5ce3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e16e26784601425c9883acd91f65becc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:46.783125 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35bfe8-628b-438a-9698-40bad0ad5ce3 service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:46.783424 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35bfe8-628b-438a-9698-40bad0ad5ce3 service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:46.783801 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35bfe8-628b-438a-9698-40bad0ad5ce3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:46.795244 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35bfe8-628b-438a-9698-40bad0ad5ce3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:46.797775 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35bfe8-628b-438a-9698-40bad0ad5ce3 service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:46.807982 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1422/2866] 10.222.0.121 () {60 vars in 1389 bytes} [Thu Aug 4 10:50:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:50:55.579520 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1b5a8-3ef5-4aeb-9fb3-1315126df6c5 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:55.580268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1b5a8-3ef5-4aeb-9fb3-1315126df6c5 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:55.580268 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1b5a8-3ef5-4aeb-9fb3-1315126df6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:55.601034 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-00c1b5a8-3ef5-4aeb-9fb3-1315126df6c5 None None] MFA Rules not processed for user `26ad0a8e94ab4bcfa8b01dc16ad08b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:55.602391 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-00c1b5a8-3ef5-4aeb-9fb3-1315126df6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:55.611253 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1445/2867] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:50:55 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:50:55.925412 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35bfe8-628b-438a-9698-40bad0ad5ce3 service swift] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:55.927018 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5d35bfe8-628b-438a-9698-40bad0ad5ce3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:55.948479 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:50:55.989101 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35bfe8-628b-438a-9698-40bad0ad5ce3 service swift] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:55.993108 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1d457-9ea8-4e23-8cdd-9adb386cbda4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:55.993743 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1d457-9ea8-4e23-8cdd-9adb386cbda4 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:55.994034 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1d457-9ea8-4e23-8cdd-9adb386cbda4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:55.994390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1d457-9ea8-4e23-8cdd-9adb386cbda4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:55.996329 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e3d1d457-9ea8-4e23-8cdd-9adb386cbda4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:56.008916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d1d457-9ea8-4e23-8cdd-9adb386cbda4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:56.010418 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d1d457-9ea8-4e23-8cdd-9adb386cbda4 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:56.020400 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1423/2868] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:50:55 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:50:56.390508 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1b5a8-3ef5-4aeb-9fb3-1315126df6c5 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:56.392285 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-00c1b5a8-3ef5-4aeb-9fb3-1315126df6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:56.468852 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1b5a8-3ef5-4aeb-9fb3-1315126df6c5 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:56.473802 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b84c4-f222-4f6f-8482-72379c77f379 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c7cfbe911c4378857848966f4f8d98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:56.474474 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b84c4-f222-4f6f-8482-72379c77f379 service placement] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:56.474755 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b84c4-f222-4f6f-8482-72379c77f379 service placement] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:56.475031 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b84c4-f222-4f6f-8482-72379c77f379 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:56.477220 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-648b84c4-f222-4f6f-8482-72379c77f379 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:56.489687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b84c4-f222-4f6f-8482-72379c77f379 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:56.491388 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b84c4-f222-4f6f-8482-72379c77f379 service placement] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:56.502402 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1446/2869] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:50:56 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:50:57.880986 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567555a-9ca0-4f7f-af68-e07c4c2decd3 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:57.881617 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567555a-9ca0-4f7f-af68-e07c4c2decd3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:57.881617 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567555a-9ca0-4f7f-af68-e07c4c2decd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:57.902969 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4567555a-9ca0-4f7f-af68-e07c4c2decd3 None None] MFA Rules not processed for user `efe6e59228bd49bfbff25a11eb682426`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:50:57.904300 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4567555a-9ca0-4f7f-af68-e07c4c2decd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:50:57.913452 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1424/2870] 10.222.0.121 () {58 vars in 1014 bytes} [Thu Aug 4 10:50:57 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:50:57.927671 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b84c4-f222-4f6f-8482-72379c77f379 service placement] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:50:57.940225 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b84c4-f222-4f6f-8482-72379c77f379 service placement] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:50:57.945684 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cae96a-b9b0-4025-bc42-5226c8fe9fa4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e16e26784601425c9883acd91f65becc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:50:57.946315 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cae96a-b9b0-4025-bc42-5226c8fe9fa4 service swift] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:50:57.946607 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cae96a-b9b0-4025-bc42-5226c8fe9fa4 service swift] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:50:57.946900 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cae96a-b9b0-4025-bc42-5226c8fe9fa4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:50:57.957319 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cae96a-b9b0-4025-bc42-5226c8fe9fa4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:50:57.958978 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cae96a-b9b0-4025-bc42-5226c8fe9fa4 service swift] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:50:57.968566 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1447/2871] 10.222.0.121 () {60 vars in 1389 bytes} [Thu Aug 4 10:50:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:51:07.896433 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=21 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:51:07.965021 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=94 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:51:14.057427 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567555a-9ca0-4f7f-af68-e07c4c2decd3 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:51:14.059204 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4567555a-9ca0-4f7f-af68-e07c4c2decd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:51:14.139135 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567555a-9ca0-4f7f-af68-e07c4c2decd3 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:51:14.145568 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df6487-c701-4cc5-8b7f-e133995245bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:51:14.146150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df6487-c701-4cc5-8b7f-e133995245bb service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:51:14.146437 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df6487-c701-4cc5-8b7f-e133995245bb service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:51:14.146710 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df6487-c701-4cc5-8b7f-e133995245bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:51:14.148587 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e5df6487-c701-4cc5-8b7f-e133995245bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:51:14.162233 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df6487-c701-4cc5-8b7f-e133995245bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:51:14.163914 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df6487-c701-4cc5-8b7f-e133995245bb service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:51:14.175159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1425/2872] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:51:14 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:51:24.170724 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=34 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:51:27.529673 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cae96a-b9b0-4025-bc42-5226c8fe9fa4 service swift] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:51:27.544156 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cae96a-b9b0-4025-bc42-5226c8fe9fa4 service swift] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:51:27.551046 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f7684-1344-4c78-b7d4-b42b124c07a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:51:27.551837 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f7684-1344-4c78-b7d4-b42b124c07a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:51:27.552149 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f7684-1344-4c78-b7d4-b42b124c07a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:51:27.552424 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f7684-1344-4c78-b7d4-b42b124c07a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:51:27.563302 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f7684-1344-4c78-b7d4-b42b124c07a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:51:27.564793 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f7684-1344-4c78-b7d4-b42b124c07a2 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:51:27.575133 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1448/2873] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:51:27 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:51:37.570349 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=6 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:52:02.628302 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df6487-c701-4cc5-8b7f-e133995245bb service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:52:02.630026 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e5df6487-c701-4cc5-8b7f-e133995245bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:52:02.729527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df6487-c701-4cc5-8b7f-e133995245bb service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:52:02.736268 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b195cff-e901-4562-8ae0-756f0bfe35e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c7cfbe911c4378857848966f4f8d98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:52:02.736924 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b195cff-e901-4562-8ae0-756f0bfe35e3 service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:52:02.737506 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b195cff-e901-4562-8ae0-756f0bfe35e3 service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:52:02.737506 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b195cff-e901-4562-8ae0-756f0bfe35e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:52:02.740030 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4b195cff-e901-4562-8ae0-756f0bfe35e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:52:02.754162 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b195cff-e901-4562-8ae0-756f0bfe35e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:52:02.755803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b195cff-e901-4562-8ae0-756f0bfe35e3 service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:52:02.767254 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1426/2874] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:52:02 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:52:12.762601 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=37 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:53:45.976932 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f7684-1344-4c78-b7d4-b42b124c07a2 service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:53:46.009123 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f7684-1344-4c78-b7d4-b42b124c07a2 service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:53:46.013438 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8375b-f281-4dd7-a632-1220df8a8265 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:53:46.014125 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8375b-f281-4dd7-a632-1220df8a8265 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:53:46.014415 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8375b-f281-4dd7-a632-1220df8a8265 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:53:46.014696 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8375b-f281-4dd7-a632-1220df8a8265 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:53:46.016893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-3ae8375b-f281-4dd7-a632-1220df8a8265 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:53:46.057972 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8375b-f281-4dd7-a632-1220df8a8265 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:53:46.059452 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8375b-f281-4dd7-a632-1220df8a8265 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:53:46.082278 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1449/2875] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:53:45 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:53:52.231519 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b195cff-e901-4562-8ae0-756f0bfe35e3 service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:53:52.242705 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b195cff-e901-4562-8ae0-756f0bfe35e3 service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:53:52.247251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf150df8-2ec7-470f-8ebd-89c91f12f740 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:53:52.247890 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf150df8-2ec7-470f-8ebd-89c91f12f740 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:53:52.248219 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf150df8-2ec7-470f-8ebd-89c91f12f740 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:53:52.248499 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf150df8-2ec7-470f-8ebd-89c91f12f740 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:53:52.250825 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cf150df8-2ec7-470f-8ebd-89c91f12f740 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:53:52.265369 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf150df8-2ec7-470f-8ebd-89c91f12f740 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:53:52.266774 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf150df8-2ec7-470f-8ebd-89c91f12f740 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:53:52.278330 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1427/2876] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:53:52 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:53:54.655542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad302e-0561-4dfe-8fc3-0ef5e687f95e None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:53:54.656212 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad302e-0561-4dfe-8fc3-0ef5e687f95e None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:53:54.656442 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad302e-0561-4dfe-8fc3-0ef5e687f95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:53:54.687085 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-d1ad302e-0561-4dfe-8fc3-0ef5e687f95e None None] MFA Rules not processed for user `efe6e59228bd49bfbff25a11eb682426`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:53:54.688566 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d1ad302e-0561-4dfe-8fc3-0ef5e687f95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:53:54.696509 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1450/2877] 10.222.0.121 () {58 vars in 972 bytes} [Thu Aug 4 10:53:54 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:53:54.738123 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf150df8-2ec7-470f-8ebd-89c91f12f740 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:53:54.750778 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf150df8-2ec7-470f-8ebd-89c91f12f740 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:53:54.757127 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590d908-89f4-4cea-8595-a215b0eb7690 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e16e26784601425c9883acd91f65becc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:53:54.757772 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590d908-89f4-4cea-8595-a215b0eb7690 service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:53:54.758053 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590d908-89f4-4cea-8595-a215b0eb7690 service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:53:54.758330 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590d908-89f4-4cea-8595-a215b0eb7690 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:53:54.767206 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7590d908-89f4-4cea-8595-a215b0eb7690 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:53:54.768847 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7590d908-89f4-4cea-8595-a215b0eb7690 service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:53:54.777210 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1428/2878] 10.222.0.121 () {60 vars in 1389 bytes} [Thu Aug 4 10:53:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 10:54:01.612304 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad302e-0561-4dfe-8fc3-0ef5e687f95e None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:54:01.705562 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad302e-0561-4dfe-8fc3-0ef5e687f95e None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:54:01.711423 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ac95f-bd54-49e4-b538-156d00d9efed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:54:01.712146 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ac95f-bd54-49e4-b538-156d00d9efed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:01.712417 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ac95f-bd54-49e4-b538-156d00d9efed admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:01.712696 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ac95f-bd54-49e4-b538-156d00d9efed admin admin] PATH_INFO: `/v3/users/26ad0a8e94ab4bcfa8b01dc16ad08b0f` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:01.714010 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ac95f-bd54-49e4-b538-156d00d9efed admin admin] RBAC: Authorizing `identity:delete_user(user_id=26ad0a8e94ab4bcfa8b01dc16ad08b0f)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:54:01.716014 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ac95f-bd54-49e4-b538-156d00d9efed admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:54:01.765778 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c52ac95f-bd54-49e4-b538-156d00d9efed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26ad0a8e94ab4bcfa8b01dc16ad08b0f'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:54:01.766492 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c52ac95f-bd54-49e4-b538-156d00d9efed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26ad0a8e94ab4bcfa8b01dc16ad08b0f'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:54:01.778198 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c52ac95f-bd54-49e4-b538-156d00d9efed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26ad0a8e94ab4bcfa8b01dc16ad08b0f'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:54:01.782979 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:54:01.786375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c52ac95f-bd54-49e4-b538-156d00d9efed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26ad0a8e94ab4bcfa8b01dc16ad08b0f'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:54:01.795028 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c52ac95f-bd54-49e4-b538-156d00d9efed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26ad0a8e94ab4bcfa8b01dc16ad08b0f'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:54:01.801242 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1451/2879] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:54:01 2022] DELETE /identity/v3/users/26ad0a8e94ab4bcfa8b01dc16ad08b0f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:54:01.985289 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590d908-89f4-4cea-8595-a215b0eb7690 service swift] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:54:01.997605 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590d908-89f4-4cea-8595-a215b0eb7690 service swift] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:54:02.059326 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f550d9-f4f2-4a97-b66b-8860ebd661a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:54:02.059963 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f550d9-f4f2-4a97-b66b-8860ebd661a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:02.060315 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f550d9-f4f2-4a97-b66b-8860ebd661a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:02.060599 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f550d9-f4f2-4a97-b66b-8860ebd661a3 admin admin] PATH_INFO: `/v3/projects/9a6e2ee8a698471faedbb97fb2b936cd` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:02.061873 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f550d9-f4f2-4a97-b66b-8860ebd661a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a6e2ee8a698471faedbb97fb2b936cd)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:54:02.063844 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f550d9-f4f2-4a97-b66b-8860ebd661a3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:54:02.113324 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-62f550d9-f4f2-4a97-b66b-8860ebd661a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a6e2ee8a698471faedbb97fb2b936cd', 'request_id': 'req-62f550d9-f4f2-4a97-b66b-8860ebd661a3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:54:02.119534 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-62f550d9-f4f2-4a97-b66b-8860ebd661a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a6e2ee8a698471faedbb97fb2b936cd', 'request_id': 'req-62f550d9-f4f2-4a97-b66b-8860ebd661a3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:54:02.125885 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-62f550d9-f4f2-4a97-b66b-8860ebd661a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a6e2ee8a698471faedbb97fb2b936cd', 'request_id': 'req-62f550d9-f4f2-4a97-b66b-8860ebd661a3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:54:02.153098 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-62f550d9-f4f2-4a97-b66b-8860ebd661a3 admin admin] The token cache is being invalidate because project 9a6e2ee8a698471faedbb97fb2b936cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:54:02.153390 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-62f550d9-f4f2-4a97-b66b-8860ebd661a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:54:02.154191 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-62f550d9-f4f2-4a97-b66b-8860ebd661a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:54:02.155794 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1429/2880] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:54:01 2022] DELETE /identity/v3/projects/9a6e2ee8a698471faedbb97fb2b936cd => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:54:02.160509 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ac95f-bd54-49e4-b538-156d00d9efed admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:54:02.162147 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c52ac95f-bd54-49e4-b538-156d00d9efed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:54:02.176084 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ac95f-bd54-49e4-b538-156d00d9efed admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:54:02.236595 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607405be-7255-4128-99d5-f5505414c5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:54:02.237266 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607405be-7255-4128-99d5-f5505414c5c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:02.237551 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607405be-7255-4128-99d5-f5505414c5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:02.237816 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607405be-7255-4128-99d5-f5505414c5c0 admin admin] PATH_INFO: `/v3/users/68eac32080534163ad17e3a9e79cf3ef` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:02.239052 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607405be-7255-4128-99d5-f5505414c5c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68eac32080534163ad17e3a9e79cf3ef)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:54:02.241075 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607405be-7255-4128-99d5-f5505414c5c0 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:54:02.288248 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-607405be-7255-4128-99d5-f5505414c5c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68eac32080534163ad17e3a9e79cf3ef'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:54:02.288922 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-607405be-7255-4128-99d5-f5505414c5c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68eac32080534163ad17e3a9e79cf3ef'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:54:02.301375 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-607405be-7255-4128-99d5-f5505414c5c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68eac32080534163ad17e3a9e79cf3ef'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:54:02.309232 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-607405be-7255-4128-99d5-f5505414c5c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68eac32080534163ad17e3a9e79cf3ef'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:54:02.318286 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-607405be-7255-4128-99d5-f5505414c5c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68eac32080534163ad17e3a9e79cf3ef'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:54:02.325036 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1452/2881] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 10:54:02 2022] DELETE /identity/v3/users/68eac32080534163ad17e3a9e79cf3ef => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:54:02.510397 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f550d9-f4f2-4a97-b66b-8860ebd661a3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:54:02.512334 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-62f550d9-f4f2-4a97-b66b-8860ebd661a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:54:02.527118 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f550d9-f4f2-4a97-b66b-8860ebd661a3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:54:02.581943 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:54:02.596862 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b011273-b384-44fc-8384-7c44230f8eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:54:02.597998 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b011273-b384-44fc-8384-7c44230f8eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:02.598478 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b011273-b384-44fc-8384-7c44230f8eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:02.598966 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b011273-b384-44fc-8384-7c44230f8eb3 admin admin] PATH_INFO: `/v3/projects/6afb923fc7f545e0a59aa9007caea945` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:02.601020 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b011273-b384-44fc-8384-7c44230f8eb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6afb923fc7f545e0a59aa9007caea945)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:54:02.604717 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b011273-b384-44fc-8384-7c44230f8eb3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:54:02.672578 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0b011273-b384-44fc-8384-7c44230f8eb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6afb923fc7f545e0a59aa9007caea945', 'request_id': 'req-0b011273-b384-44fc-8384-7c44230f8eb3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:54:02.679005 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0b011273-b384-44fc-8384-7c44230f8eb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6afb923fc7f545e0a59aa9007caea945', 'request_id': 'req-0b011273-b384-44fc-8384-7c44230f8eb3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:54:02.684791 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0b011273-b384-44fc-8384-7c44230f8eb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6afb923fc7f545e0a59aa9007caea945', 'request_id': 'req-0b011273-b384-44fc-8384-7c44230f8eb3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:54:02.699441 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0b011273-b384-44fc-8384-7c44230f8eb3 admin admin] The token cache is being invalidate because project 6afb923fc7f545e0a59aa9007caea945 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 10:54:02.699716 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0b011273-b384-44fc-8384-7c44230f8eb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:54:02.700404 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0b011273-b384-44fc-8384-7c44230f8eb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 10:54:02.702535 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1430/2882] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 10:54:02 2022] DELETE /identity/v3/projects/6afb923fc7f545e0a59aa9007caea945 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:54:02.708613 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd39ee5-ad6e-4d75-a129-903265d8e505 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:02.709227 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd39ee5-ad6e-4d75-a129-903265d8e505 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:02.709483 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd39ee5-ad6e-4d75-a129-903265d8e505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:02.770880 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-8cd39ee5-ad6e-4d75-a129-903265d8e505 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:54:02.772158 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-8cd39ee5-ad6e-4d75-a129-903265d8e505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:54:02.802896 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:54:02.833905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1453/2883] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:54:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:54:02.842866 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a59e1-c41f-46c6-852a-f9ffdb48c46b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:02.843259 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a59e1-c41f-46c6-852a-f9ffdb48c46b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:02.843419 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a59e1-c41f-46c6-852a-f9ffdb48c46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:02.865947 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-aa3a59e1-c41f-46c6-852a-f9ffdb48c46b None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:54:02.867189 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-aa3a59e1-c41f-46c6-852a-f9ffdb48c46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:54:02.876925 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1431/2884] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:54:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:54:02.881954 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd39ee5-ad6e-4d75-a129-903265d8e505 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:54:02.892484 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd39ee5-ad6e-4d75-a129-903265d8e505 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:54:02.897617 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de115940-ff09-44fc-9202-5a62b46ef211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:54:02.898243 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de115940-ff09-44fc-9202-5a62b46ef211 admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:02.898525 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de115940-ff09-44fc-9202-5a62b46ef211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:02.898794 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de115940-ff09-44fc-9202-5a62b46ef211 admin admin] PATH_INFO: `/v3/domains` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:02.899723 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de115940-ff09-44fc-9202-5a62b46ef211 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80235) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 10:54:02.899877 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de115940-ff09-44fc-9202-5a62b46ef211 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:54:02.901512 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de115940-ff09-44fc-9202-5a62b46ef211 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:54:02.913655 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1454/2885] 10.222.0.121 () {58 vars in 1158 bytes} [Thu Aug 4 10:54:02 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 10:54:02.918642 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a59e1-c41f-46c6-852a-f9ffdb48c46b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:54:02.929378 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a59e1-c41f-46c6-852a-f9ffdb48c46b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:54:02.935080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afba153-7d36-487e-8b61-fb7224258daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:54:02.935803 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afba153-7d36-487e-8b61-fb7224258daf admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:02.936122 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afba153-7d36-487e-8b61-fb7224258daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:02.936399 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afba153-7d36-487e-8b61-fb7224258daf admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:02.937077 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afba153-7d36-487e-8b61-fb7224258daf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:54:02.938986 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afba153-7d36-487e-8b61-fb7224258daf admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:54:02.970367 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1432/2886] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:54:02 2022] POST /identity/v3/projects => generated 380 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:54:02.974998 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de115940-ff09-44fc-9202-5a62b46ef211 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:54:02.985792 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de115940-ff09-44fc-9202-5a62b46ef211 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:54:03.043357 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256bda9-3f2a-4562-badd-f6ce8aa1812d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:54:03.044041 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256bda9-3f2a-4562-badd-f6ce8aa1812d admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:03.044316 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256bda9-3f2a-4562-badd-f6ce8aa1812d admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:03.044888 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256bda9-3f2a-4562-badd-f6ce8aa1812d admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:03.045573 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256bda9-3f2a-4562-badd-f6ce8aa1812d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:54:03.047518 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256bda9-3f2a-4562-badd-f6ce8aa1812d admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:54:03.071770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1455/2887] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:54:02 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:54:03.076402 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afba153-7d36-487e-8b61-fb7224258daf admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:54:03.088659 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afba153-7d36-487e-8b61-fb7224258daf admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:54:03.094796 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31c8ed-3e35-4f08-af88-0d37bdab811b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:54:03.095442 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31c8ed-3e35-4f08-af88-0d37bdab811b admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:03.095729 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31c8ed-3e35-4f08-af88-0d37bdab811b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:03.096036 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31c8ed-3e35-4f08-af88-0d37bdab811b admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:03.096778 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da31c8ed-3e35-4f08-af88-0d37bdab811b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:54:03.098410 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da31c8ed-3e35-4f08-af88-0d37bdab811b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:54:03.111437 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1433/2888] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:54:03 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:54:03.116478 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256bda9-3f2a-4562-badd-f6ce8aa1812d admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:54:03.128951 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256bda9-3f2a-4562-badd-f6ce8aa1812d admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:54:03.134456 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989e9ea-eedc-4f92-bc64-a7f047d9e6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:54:03.135129 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989e9ea-eedc-4f92-bc64-a7f047d9e6fc admin admin] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:03.135411 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989e9ea-eedc-4f92-bc64-a7f047d9e6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:03.135687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989e9ea-eedc-4f92-bc64-a7f047d9e6fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:03.136439 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989e9ea-eedc-4f92-bc64-a7f047d9e6fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:54:03.138128 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989e9ea-eedc-4f92-bc64-a7f047d9e6fc admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:54:03.149715 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1456/2889] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:54:03 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:54:03.154609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31c8ed-3e35-4f08-af88-0d37bdab811b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:54:03.165324 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31c8ed-3e35-4f08-af88-0d37bdab811b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:54:03.170514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4366e4-1cf1-4569-9aba-60586dacd362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:54:03.171194 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4366e4-1cf1-4569-9aba-60586dacd362 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:03.171481 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4366e4-1cf1-4569-9aba-60586dacd362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:03.171760 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4366e4-1cf1-4569-9aba-60586dacd362 admin admin] PATH_INFO: `/v3/projects/400197e1b8db4603a2a036b3ee157baa/users/921f73ae6c314135a57c1561de5fd9d1/roles/1b988376d12749bd8d653baa89493c5c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:03.186942 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4366e4-1cf1-4569-9aba-60586dacd362 admin admin] RBAC: Authorizing `identity:create_grant(project_id=400197e1b8db4603a2a036b3ee157baa, user_id=921f73ae6c314135a57c1561de5fd9d1, role_id=1b988376d12749bd8d653baa89493c5c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:54:03.189597 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4366e4-1cf1-4569-9aba-60586dacd362 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:54:03.204826 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1434/2890] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:54:03 2022] PUT /identity/v3/projects/400197e1b8db4603a2a036b3ee157baa/users/921f73ae6c314135a57c1561de5fd9d1/roles/1b988376d12749bd8d653baa89493c5c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:54:03.211126 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f65b5a-3344-4b8f-98ff-2f7a0a841558 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:03.211499 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f65b5a-3344-4b8f-98ff-2f7a0a841558 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:03.211785 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f65b5a-3344-4b8f-98ff-2f7a0a841558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:03.232784 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-78f65b5a-3344-4b8f-98ff-2f7a0a841558 None None] MFA Rules not processed for user `4d1eaa7a05e745a5817149d8a56dab58`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:54:03.234026 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-78f65b5a-3344-4b8f-98ff-2f7a0a841558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:54:03.299542 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1457/2891] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:54:03 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:54:03.310015 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4366e4-1cf1-4569-9aba-60586dacd362 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:54:03.312211 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cf4366e4-1cf1-4569-9aba-60586dacd362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:54:03.386420 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4366e4-1cf1-4569-9aba-60586dacd362 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:54:03.390910 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74e169-5618-46f3-98d5-bd00834093ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:54:03.391545 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74e169-5618-46f3-98d5-bd00834093ab service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:03.391868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74e169-5618-46f3-98d5-bd00834093ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:03.392359 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74e169-5618-46f3-98d5-bd00834093ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:03.401828 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab74e169-5618-46f3-98d5-bd00834093ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:54:03.403297 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab74e169-5618-46f3-98d5-bd00834093ab service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:54:03.412703 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1435/2892] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:54:03 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:54:07.597141 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b486913-325b-423d-bf8c-6a00a6b49faf None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:07.597769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b486913-325b-423d-bf8c-6a00a6b49faf None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:07.597769 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b486913-325b-423d-bf8c-6a00a6b49faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:07.630567 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-8b486913-325b-423d-bf8c-6a00a6b49faf None None] MFA Rules not processed for user `921f73ae6c314135a57c1561de5fd9d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:54:07.632074 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-8b486913-325b-423d-bf8c-6a00a6b49faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:54:07.668213 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:54:07.718307 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1458/2893] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:54:07 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:54:07.723438 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74e169-5618-46f3-98d5-bd00834093ab service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:54:07.735016 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74e169-5618-46f3-98d5-bd00834093ab service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:54:07.740184 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8db2a-84c5-4cfe-b74c-80dd8beac30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:54:07.740832 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8db2a-84c5-4cfe-b74c-80dd8beac30a admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:07.741118 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8db2a-84c5-4cfe-b74c-80dd8beac30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:07.741395 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8db2a-84c5-4cfe-b74c-80dd8beac30a admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:07.742085 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8db2a-84c5-4cfe-b74c-80dd8beac30a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:54:07.744050 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8db2a-84c5-4cfe-b74c-80dd8beac30a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:54:07.754868 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:54:07.776225 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1436/2894] 10.222.0.121 () {60 vars in 1158 bytes} [Thu Aug 4 10:54:07 2022] POST /identity/v3/projects => generated 380 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 10:54:07.780699 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b486913-325b-423d-bf8c-6a00a6b49faf None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:54:07.792545 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b486913-325b-423d-bf8c-6a00a6b49faf None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:54:07.853471 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c86ad8-ea86-47d8-82fa-5127e838bc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:54:07.854131 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c86ad8-ea86-47d8-82fa-5127e838bc6d admin admin] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:07.854413 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c86ad8-ea86-47d8-82fa-5127e838bc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:07.854687 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c86ad8-ea86-47d8-82fa-5127e838bc6d admin admin] PATH_INFO: `/v3/users` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:07.855379 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c86ad8-ea86-47d8-82fa-5127e838bc6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:54:07.857517 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c86ad8-ea86-47d8-82fa-5127e838bc6d admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:54:07.882642 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1459/2895] 10.222.0.121 () {60 vars in 1149 bytes} [Thu Aug 4 10:54:07 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 10:54:07.886967 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8db2a-84c5-4cfe-b74c-80dd8beac30a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:54:07.898351 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8db2a-84c5-4cfe-b74c-80dd8beac30a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:54:07.903904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae679f6d-9a33-46db-acb5-42ac26a0f162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:54:07.906791 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae679f6d-9a33-46db-acb5-42ac26a0f162 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:07.907084 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae679f6d-9a33-46db-acb5-42ac26a0f162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:07.907428 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae679f6d-9a33-46db-acb5-42ac26a0f162 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:07.908361 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae679f6d-9a33-46db-acb5-42ac26a0f162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:54:07.910104 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae679f6d-9a33-46db-acb5-42ac26a0f162 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:54:07.921864 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1437/2896] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:54:07 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:54:07.926662 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c86ad8-ea86-47d8-82fa-5127e838bc6d admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:54:07.937706 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c86ad8-ea86-47d8-82fa-5127e838bc6d admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:54:07.942893 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea075b07-ebfd-4ce5-a7a0-62f0f6655315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:54:07.943598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea075b07-ebfd-4ce5-a7a0-62f0f6655315 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:07.943884 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea075b07-ebfd-4ce5-a7a0-62f0f6655315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:07.944207 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea075b07-ebfd-4ce5-a7a0-62f0f6655315 admin admin] PATH_INFO: `/v3/projects/8468a55484b9490aaf0e9977e380020c/users/6dd56528a4c84be2a120cf3b6ba28823/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:07.961768 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea075b07-ebfd-4ce5-a7a0-62f0f6655315 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8468a55484b9490aaf0e9977e380020c, user_id=6dd56528a4c84be2a120cf3b6ba28823, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:54:07.964628 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea075b07-ebfd-4ce5-a7a0-62f0f6655315 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:54:07.980586 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1460/2897] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:54:07 2022] PUT /identity/v3/projects/8468a55484b9490aaf0e9977e380020c/users/6dd56528a4c84be2a120cf3b6ba28823/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:54:07.985060 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae679f6d-9a33-46db-acb5-42ac26a0f162 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:54:07.996298 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae679f6d-9a33-46db-acb5-42ac26a0f162 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:54:08.063660 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b698e-a6b8-424d-895d-d4f71e1cbafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:54:08.064227 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b698e-a6b8-424d-895d-d4f71e1cbafb admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:08.064601 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b698e-a6b8-424d-895d-d4f71e1cbafb admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:08.064880 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b698e-a6b8-424d-895d-d4f71e1cbafb admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:08.065592 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b698e-a6b8-424d-895d-d4f71e1cbafb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:54:08.067310 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b698e-a6b8-424d-895d-d4f71e1cbafb admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:54:08.079102 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1438/2898] 10.222.0.121 () {58 vars in 1127 bytes} [Thu Aug 4 10:54:07 2022] GET /identity/v3/roles => generated 1427 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 10:54:08.083259 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea075b07-ebfd-4ce5-a7a0-62f0f6655315 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:54:08.096390 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea075b07-ebfd-4ce5-a7a0-62f0f6655315 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:54:08.101825 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45afa4-a245-4e1a-9435-d858769e0be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:54:08.102541 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45afa4-a245-4e1a-9435-d858769e0be1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:08.102834 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45afa4-a245-4e1a-9435-d858769e0be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:08.103119 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45afa4-a245-4e1a-9435-d858769e0be1 admin admin] PATH_INFO: `/v3/projects/8468a55484b9490aaf0e9977e380020c/users/6dd56528a4c84be2a120cf3b6ba28823/roles/7342106a255b4bce8f150726602a8b95` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:08.105421 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc45afa4-a245-4e1a-9435-d858769e0be1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8468a55484b9490aaf0e9977e380020c, user_id=6dd56528a4c84be2a120cf3b6ba28823, role_id=7342106a255b4bce8f150726602a8b95)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:54:08.108602 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc45afa4-a245-4e1a-9435-d858769e0be1 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:54:08.120174 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1461/2899] 10.222.0.121 () {60 vars in 1488 bytes} [Thu Aug 4 10:54:08 2022] PUT /identity/v3/projects/8468a55484b9490aaf0e9977e380020c/users/6dd56528a4c84be2a120cf3b6ba28823/roles/7342106a255b4bce8f150726602a8b95 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 10:54:12.761201 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0d4d9-3d37-4136-9b65-b04e98bc0c4b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:12.762038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0d4d9-3d37-4136-9b65-b04e98bc0c4b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:12.762038 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0d4d9-3d37-4136-9b65-b04e98bc0c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:12.808450 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f7a0d4d9-3d37-4136-9b65-b04e98bc0c4b None None] MFA Rules not processed for user `6dd56528a4c84be2a120cf3b6ba28823`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:54:12.816904 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f7a0d4d9-3d37-4136-9b65-b04e98bc0c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:54:12.959996 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1439/2900] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:54:12 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:54:12.979844 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45afa4-a245-4e1a-9435-d858769e0be1 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:54:12.982480 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-cc45afa4-a245-4e1a-9435-d858769e0be1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:54:13.067791 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45afa4-a245-4e1a-9435-d858769e0be1 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:54:13.073511 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44fd16-cf04-4951-b7ba-93530fbde41f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:54:13.074173 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44fd16-cf04-4951-b7ba-93530fbde41f service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:13.074457 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44fd16-cf04-4951-b7ba-93530fbde41f service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:13.074733 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44fd16-cf04-4951-b7ba-93530fbde41f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:13.097872 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:54:13.155446 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f44fd16-cf04-4951-b7ba-93530fbde41f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:54:13.157140 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f44fd16-cf04-4951-b7ba-93530fbde41f service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:54:13.167563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1462/2901] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:54:12 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:54:22.952775 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=72,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:54:23.163598 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=18 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:54:36.011988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a3a98-4f1a-4466-a601-69e3663f1086 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:36.012690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a3a98-4f1a-4466-a601-69e3663f1086 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:36.012690 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a3a98-4f1a-4466-a601-69e3663f1086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:36.037724 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-fc3a3a98-4f1a-4466-a601-69e3663f1086 None None] MFA Rules not processed for user `921f73ae6c314135a57c1561de5fd9d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:54:36.039558 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fc3a3a98-4f1a-4466-a601-69e3663f1086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:54:36.052609 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1440/2902] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:54:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:54:43.230576 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44fd16-cf04-4951-b7ba-93530fbde41f service nova] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:54:43.232222 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-4f44fd16-cf04-4951-b7ba-93530fbde41f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:54:43.297143 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44fd16-cf04-4951-b7ba-93530fbde41f service nova] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:54:43.301479 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b9209-8a7f-4334-bc56-df2925f60f4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:54:43.302600 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b9209-8a7f-4334-bc56-df2925f60f4c service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:43.303143 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b9209-8a7f-4334-bc56-df2925f60f4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:43.303497 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b9209-8a7f-4334-bc56-df2925f60f4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:43.306172 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6e8b9209-8a7f-4334-bc56-df2925f60f4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:54:43.317725 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b9209-8a7f-4334-bc56-df2925f60f4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:54:43.319197 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b9209-8a7f-4334-bc56-df2925f60f4c service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:54:43.329024 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1463/2903] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:54:43 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:54:46.030137 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:54:53.324427 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=26 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:54:58.187436 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a3a98-4f1a-4466-a601-69e3663f1086 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:54:58.198207 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a3a98-4f1a-4466-a601-69e3663f1086 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:54:58.202115 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21b6b0-005a-4059-8c76-b47aa1e29d5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:54:58.202745 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21b6b0-005a-4059-8c76-b47aa1e29d5f service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:58.203027 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21b6b0-005a-4059-8c76-b47aa1e29d5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:58.203301 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21b6b0-005a-4059-8c76-b47aa1e29d5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:58.213443 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21b6b0-005a-4059-8c76-b47aa1e29d5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:54:58.214926 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21b6b0-005a-4059-8c76-b47aa1e29d5f service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:54:58.224912 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1441/2904] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:54:58 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:54:59.889446 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b9209-8a7f-4334-bc56-df2925f60f4c service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:54:59.903452 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b9209-8a7f-4334-bc56-df2925f60f4c service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:54:59.909304 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c530c624-47da-44ca-b772-9996acf79498 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:54:59.909965 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530c624-47da-44ca-b772-9996acf79498 service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:54:59.910322 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530c624-47da-44ca-b772-9996acf79498 service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:54:59.910611 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530c624-47da-44ca-b772-9996acf79498 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:54:59.912622 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c530c624-47da-44ca-b772-9996acf79498 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:54:59.926710 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c530c624-47da-44ca-b772-9996acf79498 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:54:59.928137 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c530c624-47da-44ca-b772-9996acf79498 service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:54:59.940668 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1464/2905] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:54:59 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:55:08.220511 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=6 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:55:09.935485 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=6 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:57:07.649401 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21b6b0-005a-4059-8c76-b47aa1e29d5f service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:57:07.651876 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cc21b6b0-005a-4059-8c76-b47aa1e29d5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:57:07.728988 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21b6b0-005a-4059-8c76-b47aa1e29d5f service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:57:07.734940 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670c70c-b6d0-4268-a20d-a491c4ff3638 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c7cfbe911c4378857848966f4f8d98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:57:07.735590 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670c70c-b6d0-4268-a20d-a491c4ff3638 service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:57:07.735877 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670c70c-b6d0-4268-a20d-a491c4ff3638 service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:57:07.736204 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670c70c-b6d0-4268-a20d-a491c4ff3638 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:57:07.738233 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1670c70c-b6d0-4268-a20d-a491c4ff3638 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:57:07.750496 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670c70c-b6d0-4268-a20d-a491c4ff3638 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:57:07.752150 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670c70c-b6d0-4268-a20d-a491c4ff3638 service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:57:07.762569 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1442/2906] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:57:07 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:57:17.566362 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c530c624-47da-44ca-b772-9996acf79498 service nova] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:57:17.568053 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-c530c624-47da-44ca-b772-9996acf79498 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:57:17.583759 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c530c624-47da-44ca-b772-9996acf79498 service nova] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:57:17.593664 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91c743-eb6f-46f7-b585-b7bf96faa4b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:57:17.594686 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91c743-eb6f-46f7-b585-b7bf96faa4b3 service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:57:17.595147 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91c743-eb6f-46f7-b585-b7bf96faa4b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:57:17.595470 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91c743-eb6f-46f7-b585-b7bf96faa4b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:57:17.597915 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-8a91c743-eb6f-46f7-b585-b7bf96faa4b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:57:17.612148 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91c743-eb6f-46f7-b585-b7bf96faa4b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:57:17.613786 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91c743-eb6f-46f7-b585-b7bf96faa4b3 service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:57:17.628619 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1465/2907] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:57:17 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:57:17.758086 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=32 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:57:19.114588 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5af5f7-dc5a-4103-832f-e24db637d576 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:57:19.115310 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5af5f7-dc5a-4103-832f-e24db637d576 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:57:19.115310 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5af5f7-dc5a-4103-832f-e24db637d576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:57:19.136663 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-fa5af5f7-dc5a-4103-832f-e24db637d576 None None] MFA Rules not processed for user `921f73ae6c314135a57c1561de5fd9d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:57:19.137914 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fa5af5f7-dc5a-4103-832f-e24db637d576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:57:19.146640 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1443/2908] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:57:19 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:57:19.514049 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91c743-eb6f-46f7-b585-b7bf96faa4b3 service nova] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:57:19.525473 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91c743-eb6f-46f7-b585-b7bf96faa4b3 service nova] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:57:19.530009 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038e82f-51f7-48cc-8344-a0f7575616f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:57:19.530683 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038e82f-51f7-48cc-8344-a0f7575616f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:57:19.530969 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038e82f-51f7-48cc-8344-a0f7575616f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:57:19.531285 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038e82f-51f7-48cc-8344-a0f7575616f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:57:19.533358 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6038e82f-51f7-48cc-8344-a0f7575616f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:57:19.544565 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6038e82f-51f7-48cc-8344-a0f7575616f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:57:19.546030 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6038e82f-51f7-48cc-8344-a0f7575616f0 service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:57:19.555228 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1466/2909] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:57:19 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:57:20.055544 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5af5f7-dc5a-4103-832f-e24db637d576 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:57:20.070342 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5af5f7-dc5a-4103-832f-e24db637d576 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:57:20.076254 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1e05f-5a85-4ebf-bdfb-7ba154ea7cdc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c7cfbe911c4378857848966f4f8d98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:57:20.076923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1e05f-5a85-4ebf-bdfb-7ba154ea7cdc service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:57:20.077191 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1e05f-5a85-4ebf-bdfb-7ba154ea7cdc service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:57:20.077688 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1e05f-5a85-4ebf-bdfb-7ba154ea7cdc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:57:20.079929 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7ef1e05f-5a85-4ebf-bdfb-7ba154ea7cdc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:57:20.093048 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef1e05f-5a85-4ebf-bdfb-7ba154ea7cdc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:57:20.094629 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef1e05f-5a85-4ebf-bdfb-7ba154ea7cdc service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:57:20.105916 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1444/2910] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:57:20 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:57:29.552086 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=12 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:57:30.101159 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=9 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:57:33.925223 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038e82f-51f7-48cc-8344-a0f7575616f0 service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:57:33.940052 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038e82f-51f7-48cc-8344-a0f7575616f0 service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:57:33.945982 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21fa3bc-68e0-4c57-8ce4-fd5a8c7c584d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:57:33.946707 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fa3bc-68e0-4c57-8ce4-fd5a8c7c584d service nova] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:57:33.946992 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fa3bc-68e0-4c57-8ce4-fd5a8c7c584d service nova] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:57:33.947262 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fa3bc-68e0-4c57-8ce4-fd5a8c7c584d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:57:33.949254 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-e21fa3bc-68e0-4c57-8ce4-fd5a8c7c584d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:57:33.964081 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21fa3bc-68e0-4c57-8ce4-fd5a8c7c584d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:57:33.965718 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21fa3bc-68e0-4c57-8ce4-fd5a8c7c584d service nova] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:57:33.978759 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1467/2911] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:57:33 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:57:43.972551 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=6 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:59:28.599383 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1e05f-5a85-4ebf-bdfb-7ba154ea7cdc service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:59:28.601092 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7ef1e05f-5a85-4ebf-bdfb-7ba154ea7cdc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:59:28.723773 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1e05f-5a85-4ebf-bdfb-7ba154ea7cdc service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:59:28.728774 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4699230-a6f2-40bf-b57a-14c11765ecd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:59:28.729574 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4699230-a6f2-40bf-b57a-14c11765ecd6 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:59:28.729913 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4699230-a6f2-40bf-b57a-14c11765ecd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:59:28.730238 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4699230-a6f2-40bf-b57a-14c11765ecd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:59:28.742081 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4699230-a6f2-40bf-b57a-14c11765ecd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:59:28.743737 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4699230-a6f2-40bf-b57a-14c11765ecd6 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:59:28.756136 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1445/2912] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:59:28 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:59:38.750114 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=35 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:59:40.151451 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6837c13-0ad8-49be-be05-15e34bf75c33 None None] REQUEST_METHOD: `POST` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:59:40.152277 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6837c13-0ad8-49be-be05-15e34bf75c33 None None] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:59:40.152277 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6837c13-0ad8-49be-be05-15e34bf75c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:59:40.178737 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.auth.core [None req-f6837c13-0ad8-49be-be05-15e34bf75c33 None None] MFA Rules not processed for user `921f73ae6c314135a57c1561de5fd9d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80235) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 10:59:40.180379 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-f6837c13-0ad8-49be-be05-15e34bf75c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:59:40.190031 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1468/2913] 10.222.0.121 () {58 vars in 963 bytes} [Thu Aug 4 10:59:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 10:59:40.239777 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4699230-a6f2-40bf-b57a-14c11765ecd6 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:59:40.250366 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4699230-a6f2-40bf-b57a-14c11765ecd6 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:59:40.254811 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a0b71-8fb1-479f-9d30-525537e299f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c19bb76d0e2544299916007fbfa62578', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:59:40.255492 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a0b71-8fb1-479f-9d30-525537e299f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:59:40.255782 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a0b71-8fb1-479f-9d30-525537e299f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:59:40.256118 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a0b71-8fb1-479f-9d30-525537e299f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:59:40.258565 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-015a0b71-8fb1-479f-9d30-525537e299f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:59:40.274981 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a0b71-8fb1-479f-9d30-525537e299f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:59:40.276427 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a0b71-8fb1-479f-9d30-525537e299f2 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:59:40.290285 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1446/2914] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:59:40 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:59:41.230374 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6837c13-0ad8-49be-be05-15e34bf75c33 None None] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:59:41.242192 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6837c13-0ad8-49be-be05-15e34bf75c33 None None] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:59:41.246828 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd54f78-4fd8-4588-93ad-218a74a706ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:59:41.247493 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd54f78-4fd8-4588-93ad-218a74a706ee service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:59:41.247781 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd54f78-4fd8-4588-93ad-218a74a706ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:59:41.248088 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd54f78-4fd8-4588-93ad-218a74a706ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:59:41.250297 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-7bd54f78-4fd8-4588-93ad-218a74a706ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 10:59:41.274535 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd54f78-4fd8-4588-93ad-218a74a706ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:59:41.277392 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd54f78-4fd8-4588-93ad-218a74a706ee service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:59:41.298983 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1469/2915] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:59:41 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:59:47.384527 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a0b71-8fb1-479f-9d30-525537e299f2 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 10:59:47.402726 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a0b71-8fb1-479f-9d30-525537e299f2 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 10:59:47.409149 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f9097-0f3f-4642-a9c7-854f2fa629d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 10:59:47.410554 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f9097-0f3f-4642-a9c7-854f2fa629d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 10:59:47.411357 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f9097-0f3f-4642-a9c7-854f2fa629d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 10:59:47.412459 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f9097-0f3f-4642-a9c7-854f2fa629d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 10:59:47.431255 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f9097-0f3f-4642-a9c7-854f2fa629d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 10:59:47.432881 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f9097-0f3f-4642-a9c7-854f2fa629d2 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 10:59:47.443805 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1447/2916] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 10:59:47 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 10:59:51.289111 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=9 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 10:59:57.438651 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=12 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 11:00:04.458999 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd54f78-4fd8-4588-93ad-218a74a706ee service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 11:00:04.469107 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd54f78-4fd8-4588-93ad-218a74a706ee service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 11:00:04.473994 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82ca61-7f19-4d28-b8d7-0be18f3faf8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 11:00:04.474799 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82ca61-7f19-4d28-b8d7-0be18f3faf8e service neutron] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 11:00:04.475131 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82ca61-7f19-4d28-b8d7-0be18f3faf8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 11:00:04.475575 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82ca61-7f19-4d28-b8d7-0be18f3faf8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 11:00:04.487114 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f82ca61-7f19-4d28-b8d7-0be18f3faf8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 11:00:04.488981 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f82ca61-7f19-4d28-b8d7-0be18f3faf8e service neutron] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 11:00:04.500931 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1470/2917] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 11:00:04 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 11:00:06.384251 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f9097-0f3f-4642-a9c7-854f2fa629d2 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 11:00:06.406678 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f9097-0f3f-4642-a9c7-854f2fa629d2 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 11:00:06.416308 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3277c8-ceef-4d36-9199-257d139206a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d947aeb749546309a8d4b7938087ba0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 11:00:06.417217 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3277c8-ceef-4d36-9199-257d139206a5 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 11:00:06.417573 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3277c8-ceef-4d36-9199-257d139206a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 11:00:06.417923 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3277c8-ceef-4d36-9199-257d139206a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 11:00:06.429908 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3277c8-ceef-4d36-9199-257d139206a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 11:00:06.431535 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3277c8-ceef-4d36-9199-257d139206a5 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 11:00:06.453235 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1448/2918] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 11:00:06 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 11:00:14.495458 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=6 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 11:00:16.444405 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=6 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 11:02:07.173757 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82ca61-7f19-4d28-b8d7-0be18f3faf8e service neutron] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 11:02:07.176265 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6f82ca61-7f19-4d28-b8d7-0be18f3faf8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 11:02:07.265204 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82ca61-7f19-4d28-b8d7-0be18f3faf8e service neutron] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 11:02:07.271325 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb143ac8-1d55-40e4-bcca-0b8a24194bb9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c7cfbe911c4378857848966f4f8d98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 11:02:07.272016 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb143ac8-1d55-40e4-bcca-0b8a24194bb9 service placement] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 11:02:07.272311 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb143ac8-1d55-40e4-bcca-0b8a24194bb9 service placement] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 11:02:07.272600 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb143ac8-1d55-40e4-bcca-0b8a24194bb9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 11:02:07.283088 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb143ac8-1d55-40e4-bcca-0b8a24194bb9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 11:02:07.284779 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb143ac8-1d55-40e4-bcca-0b8a24194bb9 service placement] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 11:02:07.299218 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1471/2919] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 11:02:07 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 11:02:09.710276 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3277c8-ceef-4d36-9199-257d139206a5 service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 11:02:09.721382 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3277c8-ceef-4d36-9199-257d139206a5 service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 11:02:09.725514 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0607442-b6a8-4da8-95da-d41da34bb69a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c7eaf4500f4ef496ba72405522006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 11:02:09.726156 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0607442-b6a8-4da8-95da-d41da34bb69a service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 11:02:09.726433 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0607442-b6a8-4da8-95da-d41da34bb69a service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 11:02:09.726689 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0607442-b6a8-4da8-95da-d41da34bb69a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 11:02:09.793418 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0607442-b6a8-4da8-95da-d41da34bb69a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 11:02:09.794870 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0607442-b6a8-4da8-95da-d41da34bb69a service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 11:02:09.805360 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1449/2920] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 11:02:09 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 11:02:13.305507 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb143ac8-1d55-40e4-bcca-0b8a24194bb9 service placement] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 11:02:13.318866 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb143ac8-1d55-40e4-bcca-0b8a24194bb9 service placement] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 11:02:13.326299 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824544d-9034-40cd-b5da-3a0b8b1f3193 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c7cfbe911c4378857848966f4f8d98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 11:02:13.327469 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824544d-9034-40cd-b5da-3a0b8b1f3193 service placement] REQUEST_METHOD: `GET` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 11:02:13.327869 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824544d-9034-40cd-b5da-3a0b8b1f3193 service placement] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 11:02:13.328913 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824544d-9034-40cd-b5da-3a0b8b1f3193 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 11:02:13.331580 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-d824544d-9034-40cd-b5da-3a0b8b1f3193 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 11:02:13.350638 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824544d-9034-40cd-b5da-3a0b8b1f3193 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 11:02:13.353563 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824544d-9034-40cd-b5da-3a0b8b1f3193 service placement] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 11:02:13.366555 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1472/2921] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 11:02:13 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 11:02:16.644486 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0607442-b6a8-4da8-95da-d41da34bb69a service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 11:02:16.655063 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0607442-b6a8-4da8-95da-d41da34bb69a service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 11:02:16.660103 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd6ff6-3e0a-41df-a7ea-4ea3614647fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 11:02:16.660794 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd6ff6-3e0a-41df-a7ea-4ea3614647fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 11:02:16.661079 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd6ff6-3e0a-41df-a7ea-4ea3614647fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 11:02:16.661380 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd6ff6-3e0a-41df-a7ea-4ea3614647fe admin admin] PATH_INFO: `/v3/users/921f73ae6c314135a57c1561de5fd9d1` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 11:02:16.662847 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd6ff6-3e0a-41df-a7ea-4ea3614647fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=921f73ae6c314135a57c1561de5fd9d1)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 11:02:16.664919 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd6ff6-3e0a-41df-a7ea-4ea3614647fe admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 11:02:16.717724 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cccd6ff6-3e0a-41df-a7ea-4ea3614647fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '921f73ae6c314135a57c1561de5fd9d1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 11:02:16.725641 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cccd6ff6-3e0a-41df-a7ea-4ea3614647fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '921f73ae6c314135a57c1561de5fd9d1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 11:02:16.726417 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cccd6ff6-3e0a-41df-a7ea-4ea3614647fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '921f73ae6c314135a57c1561de5fd9d1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 11:02:16.740262 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cccd6ff6-3e0a-41df-a7ea-4ea3614647fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '921f73ae6c314135a57c1561de5fd9d1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 11:02:16.740997 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cccd6ff6-3e0a-41df-a7ea-4ea3614647fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '921f73ae6c314135a57c1561de5fd9d1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 11:02:16.765080 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1450/2922] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 11:02:16 2022] DELETE /identity/v3/users/921f73ae6c314135a57c1561de5fd9d1 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 11:02:16.958572 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824544d-9034-40cd-b5da-3a0b8b1f3193 service placement] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 11:02:16.970921 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824544d-9034-40cd-b5da-3a0b8b1f3193 service placement] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 11:02:17.034995 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c855057-5506-4072-8d78-88575362a561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 11:02:17.035905 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c855057-5506-4072-8d78-88575362a561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 11:02:17.036366 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c855057-5506-4072-8d78-88575362a561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 11:02:17.036651 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c855057-5506-4072-8d78-88575362a561 admin admin] PATH_INFO: `/v3/projects/400197e1b8db4603a2a036b3ee157baa` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 11:02:17.038215 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c855057-5506-4072-8d78-88575362a561 admin admin] RBAC: Authorizing `identity:delete_project(project_id=400197e1b8db4603a2a036b3ee157baa)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 11:02:17.040238 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c855057-5506-4072-8d78-88575362a561 admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 11:02:17.106646 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6c855057-5506-4072-8d78-88575362a561 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '400197e1b8db4603a2a036b3ee157baa', 'request_id': 'req-6c855057-5506-4072-8d78-88575362a561'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 11:02:17.117136 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6c855057-5506-4072-8d78-88575362a561 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '400197e1b8db4603a2a036b3ee157baa', 'request_id': 'req-6c855057-5506-4072-8d78-88575362a561'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 11:02:17.125185 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6c855057-5506-4072-8d78-88575362a561 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '400197e1b8db4603a2a036b3ee157baa', 'request_id': 'req-6c855057-5506-4072-8d78-88575362a561'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 11:02:17.138693 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6c855057-5506-4072-8d78-88575362a561 admin admin] The token cache is being invalidate because project 400197e1b8db4603a2a036b3ee157baa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 11:02:17.139037 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6c855057-5506-4072-8d78-88575362a561 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 11:02:17.139770 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-6c855057-5506-4072-8d78-88575362a561 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 11:02:17.141827 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1473/2923] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 11:02:16 2022] DELETE /identity/v3/projects/400197e1b8db4603a2a036b3ee157baa => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 11:02:17.146776 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd6ff6-3e0a-41df-a7ea-4ea3614647fe admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 11:02:17.148533 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cccd6ff6-3e0a-41df-a7ea-4ea3614647fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 11:02:17.164279 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd6ff6-3e0a-41df-a7ea-4ea3614647fe admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 11:02:17.232255 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a4817-eec8-4376-b3b3-527b277c9bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 11:02:17.232942 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a4817-eec8-4376-b3b3-527b277c9bbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 11:02:17.233209 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a4817-eec8-4376-b3b3-527b277c9bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 11:02:17.233476 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a4817-eec8-4376-b3b3-527b277c9bbe admin admin] PATH_INFO: `/v3/users/6dd56528a4c84be2a120cf3b6ba28823` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 11:02:17.234853 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a4817-eec8-4376-b3b3-527b277c9bbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dd56528a4c84be2a120cf3b6ba28823)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 11:02:17.236962 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a4817-eec8-4376-b3b3-527b277c9bbe admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 11:02:17.243775 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 11:02:17.297041 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6a1a4817-eec8-4376-b3b3-527b277c9bbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dd56528a4c84be2a120cf3b6ba28823'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 11:02:17.305191 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6a1a4817-eec8-4376-b3b3-527b277c9bbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dd56528a4c84be2a120cf3b6ba28823'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 11:02:17.305939 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6a1a4817-eec8-4376-b3b3-527b277c9bbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dd56528a4c84be2a120cf3b6ba28823'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 11:02:17.319751 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6a1a4817-eec8-4376-b3b3-527b277c9bbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dd56528a4c84be2a120cf3b6ba28823'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 11:02:17.320565 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6a1a4817-eec8-4376-b3b3-527b277c9bbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dd56528a4c84be2a120cf3b6ba28823'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 11:02:17.337218 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1451/2924] 10.222.0.121 () {58 vars in 1229 bytes} [Thu Aug 4 11:02:17 2022] DELETE /identity/v3/users/6dd56528a4c84be2a120cf3b6ba28823 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 11:02:17.570290 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c855057-5506-4072-8d78-88575362a561 admin admin] Authenticating user token {{(pid=80235) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 11:02:17.573815 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.fernet_utils [None req-6c855057-5506-4072-8d78-88575362a561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80235) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 11:02:17.597613 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c855057-5506-4072-8d78-88575362a561 admin admin] Validating token access rules against request {{(pid=80235) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 11:02:17.603027 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 11:02:17.695342 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d98f69-db2e-4901-9b7b-a645b0afafdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d1eaa7a05e745a5817149d8a56dab58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03f48ab98257418c836fe2e290bdf749', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80235) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 11:02:17.696478 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d98f69-db2e-4901-9b7b-a645b0afafdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 11:02:17.697109 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d98f69-db2e-4901-9b7b-a645b0afafdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 11:02:17.697465 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d98f69-db2e-4901-9b7b-a645b0afafdb admin admin] PATH_INFO: `/v3/projects/8468a55484b9490aaf0e9977e380020c` {{(pid=80235) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 11:02:17.698910 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d98f69-db2e-4901-9b7b-a645b0afafdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8468a55484b9490aaf0e9977e380020c)` {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 11:02:17.701154 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d98f69-db2e-4901-9b7b-a645b0afafdb admin admin] RBAC: Authorization granted {{(pid=80235) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 11:02:17.766656 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c8d98f69-db2e-4901-9b7b-a645b0afafdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8468a55484b9490aaf0e9977e380020c', 'request_id': 'req-c8d98f69-db2e-4901-9b7b-a645b0afafdb'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 11:02:17.775833 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c8d98f69-db2e-4901-9b7b-a645b0afafdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8468a55484b9490aaf0e9977e380020c', 'request_id': 'req-c8d98f69-db2e-4901-9b7b-a645b0afafdb'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 11:02:17.783814 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c8d98f69-db2e-4901-9b7b-a645b0afafdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8468a55484b9490aaf0e9977e380020c', 'request_id': 'req-c8d98f69-db2e-4901-9b7b-a645b0afafdb'} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 11:02:17.796590 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c8d98f69-db2e-4901-9b7b-a645b0afafdb admin admin] The token cache is being invalidate because project 8468a55484b9490aaf0e9977e380020c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80235) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 11:02:17.797049 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c8d98f69-db2e-4901-9b7b-a645b0afafdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 11:02:17.797950 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG keystone.notifications [None req-c8d98f69-db2e-4901-9b7b-a645b0afafdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80235) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 11:02:17.799649 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: [pid: 80235|app: 0|req: 1474/2925] 10.222.0.121 () {58 vars in 1238 bytes} [Thu Aug 4 11:02:17 2022] DELETE /identity/v3/projects/8468a55484b9490aaf0e9977e380020c => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 11:02:27.332843 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 11:02:27.793066 dv-cinder-852003-1-iscsi devstack@keystone.service[80235]: DEBUG dbcounter [-] [80235] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80235) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 11:07:17.626417 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a4817-eec8-4376-b3b3-527b277c9bbe admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 11:07:17.628180 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6a1a4817-eec8-4376-b3b3-527b277c9bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 11:07:17.713165 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a4817-eec8-4376-b3b3-527b277c9bbe admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 11:07:17.777009 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749c792-e56b-480e-b558-68430c8fdc2e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01c7cfbe911c4378857848966f4f8d98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22768aba760b4307aab131d1312b2bf6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 11:07:17.777646 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749c792-e56b-480e-b558-68430c8fdc2e service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 11:07:17.777930 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749c792-e56b-480e-b558-68430c8fdc2e service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 11:07:17.778206 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749c792-e56b-480e-b558-68430c8fdc2e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 11:07:17.780163 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d749c792-e56b-480e-b558-68430c8fdc2e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 11:07:17.793444 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d749c792-e56b-480e-b558-68430c8fdc2e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 11:07:17.795088 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d749c792-e56b-480e-b558-68430c8fdc2e service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 11:07:17.805467 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1452/2926] 10.222.0.121 () {60 vars in 1370 bytes} [Thu Aug 4 11:07:17 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 11:07:27.801110 dv-cinder-852003-1-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=63 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}